This presentation was presented by Art van Scheppingen at Percona Live 2017 in Santa Clara CA and covers what you need to know to effectively monitor MongoDB
MySQL Cluster (NDB) - Best Practices Percona Live 2017Severalnines
This document summarizes best practices for optimizing performance in MySQL Cluster (NDB). It discusses topics like architecture, OS tuning, stability tuning, application design, identifying bottlenecks, and tuning tricks. The core architecture of NDB Cluster is described, including its self-healing capabilities and how it handles node failures transparently. Methods for migrating data into an NDB Cluster are also provided.
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...Severalnines
This presentation by Krzysztof Książek at Percona Live 2017 in Santa Clara, California gives detailed descriptions and comparisons of the leading open source database load balancing technologies
[jLove 2020] Where is my cache architectural patterns for caching microservi...Rafał Leszko
The document discusses various architectural patterns for caching microservices, including embedded caching, embedded distributed caching, client-server caching, caching as a cloud service, sidecar caching, and reverse proxy caching. It provides examples and pros and cons of each pattern.
5 Levels of High Availability: From Multi-instance to Hybrid CloudRafał Leszko
The document discusses 5 levels of high availability for applications and services, from single instance deployments to hybrid cloud. Level 1 involves deploying multiple instances within an availability zone or region. Level 2 adds deployment across availability zones for redundancy if one zone fails. Level 3 spans multiple regions for redundancy if an entire region fails. Level 4 involves deploying across multiple cloud providers. The highest level, Level 5 hybrid cloud, provides redundancy across cloud and on-premises infrastructure. Each level increases availability but also complexity and potential latency. The document analyzes tradeoffs between consistency, latency, and functionality at each level.
Powering Microservices with Docker, Kubernetes, Kafka, and MongoDBMongoDB
Speaker: Andrew Morgan, Principal Product Marketing Manager, MongoDB
Level: 100 (Beginner)
Track: Microservices
Organizations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Replicate your complete application for your development, test, operations, and support teams. This session introduces you to technologies such as Docker, Kubernetes, and Kafka, which are driving the microservices revolution. Learn about containers and orchestration – and most importantly, how to exploit them for stateful services such as MongoDB.
What You Will Learn:
- Why organizations are choosing to use microservice architectures, what benefits they deliver, and when they should - or shouldn't - be used.
- Technologies that are used to build microservices – including containers, orchestration and messaging systems.
- Why MongoDB is a good fit for microservices and what special steps need to be taken to make them work well together.
MySQL Load Balancers - MaxScale, ProxySQL, HAProxy, MySQL Router & nginx - A ...Severalnines
The document discusses various load balancing options for MySQL databases, including application connectors, reverse proxies, and SQL-aware proxies. It provides an overview of popular load balancers like HAProxy, MaxScale, ProxySQL, MySQL Router, and Nginx. These load balancers aim to improve high availability, performance, and scalability. They handle tasks like health checking, failover, read/write splitting, and query routing. The document analyzes the key features and limitations of each type of load balancer.
Automating and Managing MongoDB: An Analysis of Ops Manager vs. ClusterControlSeveralnines
In any busy operations environment, there are countless tasks to perform - some monthly, or weekly, some daily or more frequently, and some on an ad-hoc basis. And automation is key to performing fast, efficient and consistently repeatable software deployments and recovery.
There are many generic tools available, both commercial and open source, to aid with the automation of operational tasks. Some of these tools are even deployed in the database world. However, there are a small number of specialist domain-specific automation tools available also, and we are going to compare two of these products: MongoDB?s own Ops Manager, and ClusterControl from Severalnines.
We will cover:
* Installation and maintenance
* Complexity of architecture
* Options for redundancy
* Comparative functionality
* Monitoring, Dashboard, Alerting
* Backing up and restoring
* Automated deployment of advanced configurations
* Upgrading existing deployments
Participants should take away a clear understanding of the differences between these tools, and how they help automate and manage MongoDB operations.
From Monolith to Microservices with Cassandra, Grpc, and Falcor (Luke Tillman...DataStax
Transitioning a legacy monolithic application to microservices is a daunting task by itself and it only gets more complicated as you start to dig through all the libraries and frameworks out there meant to help. In this talk, we'll cover the transition of a real Cassandra-based application to a microservices architecture using Grpc from Google and Falcor from Netflix. (Yes, Falcor is more than just a magical luck dragon from an awesome 80's movie.) We'll talk about why these technologies were a good fit for the project as well as why Cassandra is often a great choice once you go down the path of microservices. And since all the code for the project is open source, you'll have plenty to dig into afterwards.
About the Speaker
Luke Tillman Technical Evangelist, DataStax
Luke is a Technical Evangelist for Apache Cassandra at DataStax. He's spent most of the last 15 years writing code for web applications built on relational databases both large and small. Most recently, before living the glamorous life of an Evangelist at DataStax, he worked as a software engineer at Hobsons on systems used by hundreds of colleges and universities across the U.S. and the World.
MySQL Cluster (NDB) - Best Practices Percona Live 2017Severalnines
This document summarizes best practices for optimizing performance in MySQL Cluster (NDB). It discusses topics like architecture, OS tuning, stability tuning, application design, identifying bottlenecks, and tuning tricks. The core architecture of NDB Cluster is described, including its self-healing capabilities and how it handles node failures transparently. Methods for migrating data into an NDB Cluster are also provided.
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...Severalnines
This presentation by Krzysztof Książek at Percona Live 2017 in Santa Clara, California gives detailed descriptions and comparisons of the leading open source database load balancing technologies
[jLove 2020] Where is my cache architectural patterns for caching microservi...Rafał Leszko
The document discusses various architectural patterns for caching microservices, including embedded caching, embedded distributed caching, client-server caching, caching as a cloud service, sidecar caching, and reverse proxy caching. It provides examples and pros and cons of each pattern.
5 Levels of High Availability: From Multi-instance to Hybrid CloudRafał Leszko
The document discusses 5 levels of high availability for applications and services, from single instance deployments to hybrid cloud. Level 1 involves deploying multiple instances within an availability zone or region. Level 2 adds deployment across availability zones for redundancy if one zone fails. Level 3 spans multiple regions for redundancy if an entire region fails. Level 4 involves deploying across multiple cloud providers. The highest level, Level 5 hybrid cloud, provides redundancy across cloud and on-premises infrastructure. Each level increases availability but also complexity and potential latency. The document analyzes tradeoffs between consistency, latency, and functionality at each level.
Powering Microservices with Docker, Kubernetes, Kafka, and MongoDBMongoDB
Speaker: Andrew Morgan, Principal Product Marketing Manager, MongoDB
Level: 100 (Beginner)
Track: Microservices
Organizations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Replicate your complete application for your development, test, operations, and support teams. This session introduces you to technologies such as Docker, Kubernetes, and Kafka, which are driving the microservices revolution. Learn about containers and orchestration – and most importantly, how to exploit them for stateful services such as MongoDB.
What You Will Learn:
- Why organizations are choosing to use microservice architectures, what benefits they deliver, and when they should - or shouldn't - be used.
- Technologies that are used to build microservices – including containers, orchestration and messaging systems.
- Why MongoDB is a good fit for microservices and what special steps need to be taken to make them work well together.
MySQL Load Balancers - MaxScale, ProxySQL, HAProxy, MySQL Router & nginx - A ...Severalnines
The document discusses various load balancing options for MySQL databases, including application connectors, reverse proxies, and SQL-aware proxies. It provides an overview of popular load balancers like HAProxy, MaxScale, ProxySQL, MySQL Router, and Nginx. These load balancers aim to improve high availability, performance, and scalability. They handle tasks like health checking, failover, read/write splitting, and query routing. The document analyzes the key features and limitations of each type of load balancer.
Automating and Managing MongoDB: An Analysis of Ops Manager vs. ClusterControlSeveralnines
In any busy operations environment, there are countless tasks to perform - some monthly, or weekly, some daily or more frequently, and some on an ad-hoc basis. And automation is key to performing fast, efficient and consistently repeatable software deployments and recovery.
There are many generic tools available, both commercial and open source, to aid with the automation of operational tasks. Some of these tools are even deployed in the database world. However, there are a small number of specialist domain-specific automation tools available also, and we are going to compare two of these products: MongoDB?s own Ops Manager, and ClusterControl from Severalnines.
We will cover:
* Installation and maintenance
* Complexity of architecture
* Options for redundancy
* Comparative functionality
* Monitoring, Dashboard, Alerting
* Backing up and restoring
* Automated deployment of advanced configurations
* Upgrading existing deployments
Participants should take away a clear understanding of the differences between these tools, and how they help automate and manage MongoDB operations.
From Monolith to Microservices with Cassandra, Grpc, and Falcor (Luke Tillman...DataStax
Transitioning a legacy monolithic application to microservices is a daunting task by itself and it only gets more complicated as you start to dig through all the libraries and frameworks out there meant to help. In this talk, we'll cover the transition of a real Cassandra-based application to a microservices architecture using Grpc from Google and Falcor from Netflix. (Yes, Falcor is more than just a magical luck dragon from an awesome 80's movie.) We'll talk about why these technologies were a good fit for the project as well as why Cassandra is often a great choice once you go down the path of microservices. And since all the code for the project is open source, you'll have plenty to dig into afterwards.
About the Speaker
Luke Tillman Technical Evangelist, DataStax
Luke is a Technical Evangelist for Apache Cassandra at DataStax. He's spent most of the last 15 years writing code for web applications built on relational databases both large and small. Most recently, before living the glamorous life of an Evangelist at DataStax, he worked as a software engineer at Hobsons on systems used by hundreds of colleges and universities across the U.S. and the World.
Webinar slides: How to Measure Database Availability?Severalnines
Database availability is notoriously hard to measure and report on, although it is an important KPI in any SLA between you and your customer. We often define availability in terms of 9’s (e.g. 99.9% or 99.999%), although there is often a lack of understanding of what these numbers might mean, or how we can measure them.
Is the database available if an instance is up and running, but it is unable to serve any requests? Or if response times are excessively long, so that users consider the service unusable? Is the impact of one longer outage the same as multiple shorter outages? How do partial outages affect database availability, where some users are unable to use the service while others are completely unaffected?
Not agreeing on precise definitions with your customer might lead to dissatisfaction. The database team might be reporting that they have met their availability goals, while the customer is dissatisfied with the service. In this webinar, we will discuss the different factors that affect database availability. We will then see how you can measure your database availability in a realistic way.
AGENDA
- Defining availability targets
- Critical business functions
- Customer needs
- Duration and frequency of downtime
- Planned vs unplanned downtime
- SLA
- Measuring the database availability
- Failover/Switchover time
- Recovery time
- Upgrade time
- Queries latency
- Restoration time from backup
- Service outage time
- Instrumentation and tools to measure database availability:
- Free & open-source tools
- CC's Operational Report
- Paid tools
SPEAKER
Bartlomiej Oles is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
MySQL on Docker - Containerizing the DolphinSeveralnines
Docker is becoming more mainstream and adopted by users as a method to package and deploy self-sufficient applications in primarily stateless Linux containers. It's a great toolset on top of OS-level virtualization (LXC, a.k.a containers) and plays well in the world of micro services.
However, Docker containers are transient by default. If a container is destroyed, all data created is also lost. For a stateful service like a database, this is a major headache to say the least.
There are a number ways to provide persistent storage in Docker containers. In this presentation, we will talk about how to setup a persistence data service with Docker that can be torn down and brought up across hosts and containers.
We will touch upon orchestration tools, shared volumes, data-only-containers, security and configuration management, multi-host networking, service discovery and implications on monitoring when we move from host-centric to role-centric services with shorter life cycles.
MongoDB .local Bengaluru 2019: Using MongoDB Services in Kubernetes: Any Plat...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
MariaDB adoption in Linux distributions and development environmentsOtto Kekäläinen
Presentation given at the M|17 MariaDB User Conference 2017
https://m17.mariadb.com/
Linux, Apache, MySQL and PHP used to be the most widely used web application stacks. As technology evolves, this is no longer the case. For the M part, MariaDB has replaced MySQL in numerous Linux distributions and development environments, and is becoming the new M in most production environments as well. This talk presents how the landscape looks today, and why and how web developers are migrating to MariaDB around the globe.
This is the presentation delivered by Karthik.P.R at MySQL User Camp Bangalore on 09th June 2017. ProxySQL is a high performance MySQL Load Balancer Designed to scale database servers.
Where is my cache architectural patterns for caching microservices by exampleRafał Leszko
The document discusses various architectural patterns for caching microservices, including embedded caching, embedded distributed caching, client-server caching, cloud caching, sidecar caching, reverse proxy caching, and reverse proxy sidecar caching. It provides examples and pros and cons of each pattern. The presentation concludes with a summary of when each pattern may be best suited based on factors like whether the application is aware of the cache, if it uses containers, the data volume, security restrictions, language agnostic needs, and cloud usage.
MongoDB .local Bengaluru 2019: New Encryption Capabilities in MongoDB 4.2: A ...MongoDB
New encryption capabilities in MongoDB 4.2 provide client-side field-level encryption that protects sensitive data. The encryption is performed by the client drivers using modern cryptography, keeping encrypted fields opaque to the database server and operators. This allows individual fields to be encrypted per document with customer-managed keys. The presentation provides a history of database security methods and covers the new encryption in terms of design, cryptography used, and developer experience. A code example demonstrates how to configure and use the new encryption capabilities.
Webinar slides: How to deploy and manage HAProxy, MaxScale or ProxySQL with C...Severalnines
Proxies are building blocks of high availability setups for MySQL. They can detect failed nodes and route queries to hosts which are still available. If your master failed and you had to promote one of your slaves, proxies will detect such topology changes and route your traffic accordingly.
More advanced proxies can do much more, such as route traffic based on precise query rules, cache queries or mirror them. They can be even used to implement different types of sharding.
In this webinar we talk about support for proxies for MySQL HA setups in ClusterControl: how they differ and what their pros and cons are. And we show you how you can easily deploy and manage HAProxy, MaxScale and ProxySQL from ClusterControl during a live demo.
AGENDA
Introduction
Why use a proxy layer?
Comparison of proxies - the pros & cons
- HAProxy
- MaxScale
- ProxySQL
Live demo of proxy support in ClusterControl
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
Percona Cluster ( Galera ) is one of the best database solution that provides synchronous replication. The feature like automatic recovery, GTID and multi threaded replication makes it powerful along with ( XtraDB and Xtrabackup ).
The good solution for MySQL HA.
Performance Tuning Cheat Sheet for MongoDBSeveralnines
Bart Oles - Severalnines AB
Database performance affects organizational performance, and we tend to look for quick fixes when under stress. But how can we better understand our database workload and factors that may cause harm to it? What are the limitations in MongoDB that could potentially impact cluster performance?
In this talk, we will show you how to identify the factors that limit database performance. We will start with the free MongoDB Cloud monitoring tools. Then we will move on to log files and queries. To be able to achieve optimal use of hardware resources, we will take a look into kernel optimization and other crucial OS settings. Finally, we will look into how to examine performance of MongoDB replication.
Webinar slides: An Introduction to Performance Monitoring for PostgreSQLSeveralnines
To operate PostgreSQL efficiently, you need to have insight into database performance and make sure it is at optimal levels.
With that in mind, we dive into monitoring PostgreSQL for performance in this webinar replay.
PostgreSQL offers many metrics through various status overviews and commands, but which ones really matter to you? How do you trend and alert on them? What is the meaning behind the metrics? And what are some of the most common causes for performance problems in production?
We discuss this and more in ordinary, plain DBA language. We also have a look at some of the tools available for PostgreSQL monitoring and trending; and we’ll show you how to leverage ClusterControl’s PostgreSQL metrics, dashboards, custom alerting and other features to track and optimize the performance of your system.
AGENDA
- PostgreSQL architecture overview
- Performance problems in production
- Common causes
- Key PostgreSQL metrics and their meaning
- Tuning for performance
- Performance monitoring tools
- Impact of monitoring on performance
- How to use ClusterControl to identify performance issues
- Demo
SPEAKER
Sebastian Insausti, Support Engineer at Severalnines, has loved technology since his childhood, when he did his first computer course (Windows 3.11). And from that moment he was decided on what his profession would be. He has since built up experience with MySQL, PostgreSQL, HAProxy, WAF (ModSecurity), Linux (RedHat, CentOS, OL, Ubuntu server), Monitoring (Nagios), Networking and Virtualization (VMWare, Proxmox, Hyper-V, RHEV).
Prior to joining Severalnines, Sebastian worked as a consultant to state companies in security, database replication and high availability scenarios. He’s also a speaker and has given a few talks locally on InnoDB Cluster and MySQL Enterprise together with an Oracle team. Previous to that, he worked for a Mexican company as chief of sysadmin department as well as for a local ISP (Internet Service Provider), where he managed customers' servers and connectivity.
This webinar builds upon a related blog post by Sebastian: https://severalnines.com/blog/performance-cheat-sheet-postgresql.
This talk will explain best practices for upgrade techniques in MySQL. In deep dive, we will go over how to upgrade successfully to MySQL 8.0. Explain MySQL 8.0 upgrade specific challenges. Go over gotchas and best practices. Review the latest version of MySQL 8.0 and bug reports.
Tips to drive maria db cluster performance for nextcloudSeveralnines
200
● SSD
2000
● NVMe
4000
Tune for your hardware. Higher is better but avoid over-committing IOPS.
innodb_flush_log_at_trx_commit 1 Flush logs at each transaction commit for ACID compliance.
innodb_log_buffer_size 16M-64M Default is 8M. Increase for more transactions per second.
innodb_log_file_size 1G Default is 48M. Increase for more transactions per second.
innodb_flush_method O_DIRECT Bypass OS cache for better durability.
innodb_thread_concurrency 0 Allow InnoDB to manage thread concurrency level.
Galaxy Big Data with MariaDB 10 by Bernard Garros, Sandrine Chirokoff and Stéphane Varoqui.
Presented 26.6.2014 at the MariaDB Roadshow in Paris, France.
Webseminar: MariaDB Enterprise und MariaDB Enterprise ClusterMariaDB Corporation
This document provides information about MariaDB Enterprise and MariaDB Enterprise Cluster from Ralf Gebhardt, including:
- An agenda covering MariaDB, MariaDB Enterprise, MariaDB Enterprise Cluster, services, and more info.
- Background on MariaDB, the MariaDB Foundation, MariaDB.com, and SkySQL.
- A timeline of MariaDB releases from 5.1 to the current 10.0 and Galera Cluster 10.
- An overview of key features and optimizations in MariaDB 10 like multi-source replication and improved query optimization.
- Mention of Fusion-IO page compression providing a 30% performance increase with atomic writes.
High performance and high availability proxies for MySQLMydbops
This is presented at LSPE meet up on 17-06-2017.
ProxySQL and MaxScale are the next-generation database proxy that manages security, scalability, high availability and data streaming in scale-out deployments
ProxySQL and MaxScale have been most widely adapted in industry these days. They can be used in smaller platforms to bigger platforms. The Objective of this talk is to make awareness about the next-gen database proxies and their value addition.
HBaseCon2017 Analyzing cryptocurrencies in real time with hBase, Kafka and St...HBaseCon
Unlike other blockchain technologies that take hours to settle, Ripple can confirm and settle a transaction in seconds with a throughput of a 1000 transactions a second. Designing a real-time analytics solution needed a something that was scalable, reliable and most importantly guaranteed consistency at the speed of availability.
The combination of hBase as storage, and Kafka and Storm as the processing framework allows Ripple to publish transactional activity within milliseconds of committed transaction. Queries are funnelled through a diverse set of hand crafted API. RippleCharts is the public facing visual analytics on nodeJS(D3) that heavily leverages those API, but we also leverage the same resources to provide to our compliance analysts valuable information for investigations and research.
by Abraham Tom and Warren Anderson of Ripple
Deploying MariaDB databases with containers at Nokia NetworksMariaDB plc
Nokia is focused on providing software and products that facilitate rapid development, deployment and scaling of products and services to customers. The Common Software Foundation (CSF) within Nokia develops and supports product reuse by multiple applications within Nokia, including MariaDB. Their focus over the last year has been to develop a containerized MariaDB solution supporting multiple architectures, including both clustering and primary/secondary replication with MariaDB MaxScale. In this talk, Rick Lane discusses this journey of these containerized solutions from development to customer trials, including problems encountered and solutions.
Webinar slides: How to automate and manage MongoDB & Percona Server for MongoDBSeveralnines
Join us as the Severalnines & Percona teams join forces for a different perspective on how to automate and manage your MongoDB or Percona Server for MongoDB databases. During this webinar, we’ll walk you through the key features of the Percona Server for MongoDB as compared to MongoDB itself as well as how to manage both “flavours” of MongoDB.
There are many generic tools available, both commercial and open source, to aid with the automation of operational tasks. Some of these tools are even deployed in the database world.
However, there are a small number of specialist domain-specific automation tools available, and we are going to compare the MongoDB-relevant functionality of two of these products: MongoDB’s Ops Manager, and ClusterControl from Severalnines. Attendees should take away a clear understanding of the differences between these tools, and how they help automate and manage MongoDB operations.
AGENDA
- Introduction to Percona Server for MongoDB
- How to automate and manage MongoDB
- Installation and maintenance
- Complexity of architecture
- Options for redundancy
- Comparative functionality
- Monitoring, Dashboard, Alerting
- Backing up your deployments
- Automated deployment of advanced configurations
- Upgrading existing deployments
SPEAKERS
Ruairí Newman is passionate about all things cloud and automation and has worked for MongoDB, VMware and Amazon Web Services among others. He has a background in Operational Support Systems and Professional Services.
Prior to joining Severalnines, Ruairí worked for Huawei Ireland as Senior Cloud Solutions Architect on their Web Services project, where he advised on commodity cloud architecture and Monitoring technologies, and deployed and administered a Research & Development Openstack lab.
Prior to joining Percona as a Product Manager, Tyler Duzan spent almost 13 years as an operations and security engineer in a variety of different industries. Deciding to take his analytical mindset and strategic focus into new territory, Tyler is applying his knowledge to solving business problems for Percona customers with inventive solutions combining technology and services.
Webinar slides: DevOps Tutorial: how to automate your database infrastructureSeveralnines
Join our guest speaker Riaan Nolan of mukuru.com, the First Puppet Labs Certified Professional in South Africa, as he walks us through the facets of DevOps integrations and the mission-critical advantages that database automation can bring to your database infrastructure.
Infrastructure automation isn’t easy, but it’s not rocket science either. Done right, it is a worthwhile investment, but deciding on which tools to invest in can be a confusing and overwhelming process. Riaan will share some of his secrets on how to proceed with this and he knows what he’s talking about: he saves the companies he works for substantial amounts on their monthly IT bills, typically around 50%.
Don’t miss out on this opportunity to understand how you can find efficiencies for your database infrastructure and do watch this webinar to understand the key pain points, which indicate that it’s time to invest in database automation.
AGENDA
DevOps and databases - what are the challenges
Managing databases in a DevOps environment
- Requirements from microservice environments
- Automated deployments
- Performance monitoring
- Backups
- Schema changes
- Version upgrades
- Automated failover
- Integration with ChatOps and other tools
Data distribution
- Database hosting in cloud environments
- Managing data flows
Cloud Automation on AWS
SPEAKERS
Riaan Nolan was the First Puppet Labs Certified Professional in South Africa. Riaan uses Amazon EC2, VPC and Autoscale with Cloudformation to spin up complete stacks with Autoscaling Fleets. He saves companies substantial amounts on their monthly IT bills, typically around 50% - yes, at one company that meant $500k+ per year. And he’s participated in a number of community tech related forums. He uses next generation technologies such as AWS, Cloudformation, Autoscale, Puppet, GlusterFS, NGINX, Magento and PHP to power huge eCommerce stores. His specialties are Puppet Automation, Cloud Deployments, eCommerce, eMarketing, Specialized Linux Services, Windows, Process making, Budgets, Asset Tracking, Procurement.
- Devops Lead, Mukuru
- Expert Live Systems Administrator, foodpanda | Hellofood
- Senior Systems Administrator / Infrastructure Lead, Rocket Internet GmbH
- Senior Technology Manager, Africa Internet Accelerator
Art van Scheppingen is a Senior Support Engineer at Severalnines. He’s a pragmatic MySQL and Database expert with over 15 years experience in web development. He previously worked at Spil Games as Head of Database Engineering, where he kept a broad vision upon the whole database environment: from MySQL to Couchbase, Vertica to Hadoop and from Sphinx Search to SOLR. He regularly presents his work and projects at various conferences (Percona Live, FOSDEM) and related meetups.
Webinar slides: How to Measure Database Availability?Severalnines
Database availability is notoriously hard to measure and report on, although it is an important KPI in any SLA between you and your customer. We often define availability in terms of 9’s (e.g. 99.9% or 99.999%), although there is often a lack of understanding of what these numbers might mean, or how we can measure them.
Is the database available if an instance is up and running, but it is unable to serve any requests? Or if response times are excessively long, so that users consider the service unusable? Is the impact of one longer outage the same as multiple shorter outages? How do partial outages affect database availability, where some users are unable to use the service while others are completely unaffected?
Not agreeing on precise definitions with your customer might lead to dissatisfaction. The database team might be reporting that they have met their availability goals, while the customer is dissatisfied with the service. In this webinar, we will discuss the different factors that affect database availability. We will then see how you can measure your database availability in a realistic way.
AGENDA
- Defining availability targets
- Critical business functions
- Customer needs
- Duration and frequency of downtime
- Planned vs unplanned downtime
- SLA
- Measuring the database availability
- Failover/Switchover time
- Recovery time
- Upgrade time
- Queries latency
- Restoration time from backup
- Service outage time
- Instrumentation and tools to measure database availability:
- Free & open-source tools
- CC's Operational Report
- Paid tools
SPEAKER
Bartlomiej Oles is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
MySQL on Docker - Containerizing the DolphinSeveralnines
Docker is becoming more mainstream and adopted by users as a method to package and deploy self-sufficient applications in primarily stateless Linux containers. It's a great toolset on top of OS-level virtualization (LXC, a.k.a containers) and plays well in the world of micro services.
However, Docker containers are transient by default. If a container is destroyed, all data created is also lost. For a stateful service like a database, this is a major headache to say the least.
There are a number ways to provide persistent storage in Docker containers. In this presentation, we will talk about how to setup a persistence data service with Docker that can be torn down and brought up across hosts and containers.
We will touch upon orchestration tools, shared volumes, data-only-containers, security and configuration management, multi-host networking, service discovery and implications on monitoring when we move from host-centric to role-centric services with shorter life cycles.
MongoDB .local Bengaluru 2019: Using MongoDB Services in Kubernetes: Any Plat...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
MariaDB adoption in Linux distributions and development environmentsOtto Kekäläinen
Presentation given at the M|17 MariaDB User Conference 2017
https://m17.mariadb.com/
Linux, Apache, MySQL and PHP used to be the most widely used web application stacks. As technology evolves, this is no longer the case. For the M part, MariaDB has replaced MySQL in numerous Linux distributions and development environments, and is becoming the new M in most production environments as well. This talk presents how the landscape looks today, and why and how web developers are migrating to MariaDB around the globe.
This is the presentation delivered by Karthik.P.R at MySQL User Camp Bangalore on 09th June 2017. ProxySQL is a high performance MySQL Load Balancer Designed to scale database servers.
Where is my cache architectural patterns for caching microservices by exampleRafał Leszko
The document discusses various architectural patterns for caching microservices, including embedded caching, embedded distributed caching, client-server caching, cloud caching, sidecar caching, reverse proxy caching, and reverse proxy sidecar caching. It provides examples and pros and cons of each pattern. The presentation concludes with a summary of when each pattern may be best suited based on factors like whether the application is aware of the cache, if it uses containers, the data volume, security restrictions, language agnostic needs, and cloud usage.
MongoDB .local Bengaluru 2019: New Encryption Capabilities in MongoDB 4.2: A ...MongoDB
New encryption capabilities in MongoDB 4.2 provide client-side field-level encryption that protects sensitive data. The encryption is performed by the client drivers using modern cryptography, keeping encrypted fields opaque to the database server and operators. This allows individual fields to be encrypted per document with customer-managed keys. The presentation provides a history of database security methods and covers the new encryption in terms of design, cryptography used, and developer experience. A code example demonstrates how to configure and use the new encryption capabilities.
Webinar slides: How to deploy and manage HAProxy, MaxScale or ProxySQL with C...Severalnines
Proxies are building blocks of high availability setups for MySQL. They can detect failed nodes and route queries to hosts which are still available. If your master failed and you had to promote one of your slaves, proxies will detect such topology changes and route your traffic accordingly.
More advanced proxies can do much more, such as route traffic based on precise query rules, cache queries or mirror them. They can be even used to implement different types of sharding.
In this webinar we talk about support for proxies for MySQL HA setups in ClusterControl: how they differ and what their pros and cons are. And we show you how you can easily deploy and manage HAProxy, MaxScale and ProxySQL from ClusterControl during a live demo.
AGENDA
Introduction
Why use a proxy layer?
Comparison of proxies - the pros & cons
- HAProxy
- MaxScale
- ProxySQL
Live demo of proxy support in ClusterControl
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
Percona Cluster ( Galera ) is one of the best database solution that provides synchronous replication. The feature like automatic recovery, GTID and multi threaded replication makes it powerful along with ( XtraDB and Xtrabackup ).
The good solution for MySQL HA.
Performance Tuning Cheat Sheet for MongoDBSeveralnines
Bart Oles - Severalnines AB
Database performance affects organizational performance, and we tend to look for quick fixes when under stress. But how can we better understand our database workload and factors that may cause harm to it? What are the limitations in MongoDB that could potentially impact cluster performance?
In this talk, we will show you how to identify the factors that limit database performance. We will start with the free MongoDB Cloud monitoring tools. Then we will move on to log files and queries. To be able to achieve optimal use of hardware resources, we will take a look into kernel optimization and other crucial OS settings. Finally, we will look into how to examine performance of MongoDB replication.
Webinar slides: An Introduction to Performance Monitoring for PostgreSQLSeveralnines
To operate PostgreSQL efficiently, you need to have insight into database performance and make sure it is at optimal levels.
With that in mind, we dive into monitoring PostgreSQL for performance in this webinar replay.
PostgreSQL offers many metrics through various status overviews and commands, but which ones really matter to you? How do you trend and alert on them? What is the meaning behind the metrics? And what are some of the most common causes for performance problems in production?
We discuss this and more in ordinary, plain DBA language. We also have a look at some of the tools available for PostgreSQL monitoring and trending; and we’ll show you how to leverage ClusterControl’s PostgreSQL metrics, dashboards, custom alerting and other features to track and optimize the performance of your system.
AGENDA
- PostgreSQL architecture overview
- Performance problems in production
- Common causes
- Key PostgreSQL metrics and their meaning
- Tuning for performance
- Performance monitoring tools
- Impact of monitoring on performance
- How to use ClusterControl to identify performance issues
- Demo
SPEAKER
Sebastian Insausti, Support Engineer at Severalnines, has loved technology since his childhood, when he did his first computer course (Windows 3.11). And from that moment he was decided on what his profession would be. He has since built up experience with MySQL, PostgreSQL, HAProxy, WAF (ModSecurity), Linux (RedHat, CentOS, OL, Ubuntu server), Monitoring (Nagios), Networking and Virtualization (VMWare, Proxmox, Hyper-V, RHEV).
Prior to joining Severalnines, Sebastian worked as a consultant to state companies in security, database replication and high availability scenarios. He’s also a speaker and has given a few talks locally on InnoDB Cluster and MySQL Enterprise together with an Oracle team. Previous to that, he worked for a Mexican company as chief of sysadmin department as well as for a local ISP (Internet Service Provider), where he managed customers' servers and connectivity.
This webinar builds upon a related blog post by Sebastian: https://severalnines.com/blog/performance-cheat-sheet-postgresql.
This talk will explain best practices for upgrade techniques in MySQL. In deep dive, we will go over how to upgrade successfully to MySQL 8.0. Explain MySQL 8.0 upgrade specific challenges. Go over gotchas and best practices. Review the latest version of MySQL 8.0 and bug reports.
Tips to drive maria db cluster performance for nextcloudSeveralnines
200
● SSD
2000
● NVMe
4000
Tune for your hardware. Higher is better but avoid over-committing IOPS.
innodb_flush_log_at_trx_commit 1 Flush logs at each transaction commit for ACID compliance.
innodb_log_buffer_size 16M-64M Default is 8M. Increase for more transactions per second.
innodb_log_file_size 1G Default is 48M. Increase for more transactions per second.
innodb_flush_method O_DIRECT Bypass OS cache for better durability.
innodb_thread_concurrency 0 Allow InnoDB to manage thread concurrency level.
Galaxy Big Data with MariaDB 10 by Bernard Garros, Sandrine Chirokoff and Stéphane Varoqui.
Presented 26.6.2014 at the MariaDB Roadshow in Paris, France.
Webseminar: MariaDB Enterprise und MariaDB Enterprise ClusterMariaDB Corporation
This document provides information about MariaDB Enterprise and MariaDB Enterprise Cluster from Ralf Gebhardt, including:
- An agenda covering MariaDB, MariaDB Enterprise, MariaDB Enterprise Cluster, services, and more info.
- Background on MariaDB, the MariaDB Foundation, MariaDB.com, and SkySQL.
- A timeline of MariaDB releases from 5.1 to the current 10.0 and Galera Cluster 10.
- An overview of key features and optimizations in MariaDB 10 like multi-source replication and improved query optimization.
- Mention of Fusion-IO page compression providing a 30% performance increase with atomic writes.
High performance and high availability proxies for MySQLMydbops
This is presented at LSPE meet up on 17-06-2017.
ProxySQL and MaxScale are the next-generation database proxy that manages security, scalability, high availability and data streaming in scale-out deployments
ProxySQL and MaxScale have been most widely adapted in industry these days. They can be used in smaller platforms to bigger platforms. The Objective of this talk is to make awareness about the next-gen database proxies and their value addition.
HBaseCon2017 Analyzing cryptocurrencies in real time with hBase, Kafka and St...HBaseCon
Unlike other blockchain technologies that take hours to settle, Ripple can confirm and settle a transaction in seconds with a throughput of a 1000 transactions a second. Designing a real-time analytics solution needed a something that was scalable, reliable and most importantly guaranteed consistency at the speed of availability.
The combination of hBase as storage, and Kafka and Storm as the processing framework allows Ripple to publish transactional activity within milliseconds of committed transaction. Queries are funnelled through a diverse set of hand crafted API. RippleCharts is the public facing visual analytics on nodeJS(D3) that heavily leverages those API, but we also leverage the same resources to provide to our compliance analysts valuable information for investigations and research.
by Abraham Tom and Warren Anderson of Ripple
Deploying MariaDB databases with containers at Nokia NetworksMariaDB plc
Nokia is focused on providing software and products that facilitate rapid development, deployment and scaling of products and services to customers. The Common Software Foundation (CSF) within Nokia develops and supports product reuse by multiple applications within Nokia, including MariaDB. Their focus over the last year has been to develop a containerized MariaDB solution supporting multiple architectures, including both clustering and primary/secondary replication with MariaDB MaxScale. In this talk, Rick Lane discusses this journey of these containerized solutions from development to customer trials, including problems encountered and solutions.
Webinar slides: How to automate and manage MongoDB & Percona Server for MongoDBSeveralnines
Join us as the Severalnines & Percona teams join forces for a different perspective on how to automate and manage your MongoDB or Percona Server for MongoDB databases. During this webinar, we’ll walk you through the key features of the Percona Server for MongoDB as compared to MongoDB itself as well as how to manage both “flavours” of MongoDB.
There are many generic tools available, both commercial and open source, to aid with the automation of operational tasks. Some of these tools are even deployed in the database world.
However, there are a small number of specialist domain-specific automation tools available, and we are going to compare the MongoDB-relevant functionality of two of these products: MongoDB’s Ops Manager, and ClusterControl from Severalnines. Attendees should take away a clear understanding of the differences between these tools, and how they help automate and manage MongoDB operations.
AGENDA
- Introduction to Percona Server for MongoDB
- How to automate and manage MongoDB
- Installation and maintenance
- Complexity of architecture
- Options for redundancy
- Comparative functionality
- Monitoring, Dashboard, Alerting
- Backing up your deployments
- Automated deployment of advanced configurations
- Upgrading existing deployments
SPEAKERS
Ruairí Newman is passionate about all things cloud and automation and has worked for MongoDB, VMware and Amazon Web Services among others. He has a background in Operational Support Systems and Professional Services.
Prior to joining Severalnines, Ruairí worked for Huawei Ireland as Senior Cloud Solutions Architect on their Web Services project, where he advised on commodity cloud architecture and Monitoring technologies, and deployed and administered a Research & Development Openstack lab.
Prior to joining Percona as a Product Manager, Tyler Duzan spent almost 13 years as an operations and security engineer in a variety of different industries. Deciding to take his analytical mindset and strategic focus into new territory, Tyler is applying his knowledge to solving business problems for Percona customers with inventive solutions combining technology and services.
Webinar slides: DevOps Tutorial: how to automate your database infrastructureSeveralnines
Join our guest speaker Riaan Nolan of mukuru.com, the First Puppet Labs Certified Professional in South Africa, as he walks us through the facets of DevOps integrations and the mission-critical advantages that database automation can bring to your database infrastructure.
Infrastructure automation isn’t easy, but it’s not rocket science either. Done right, it is a worthwhile investment, but deciding on which tools to invest in can be a confusing and overwhelming process. Riaan will share some of his secrets on how to proceed with this and he knows what he’s talking about: he saves the companies he works for substantial amounts on their monthly IT bills, typically around 50%.
Don’t miss out on this opportunity to understand how you can find efficiencies for your database infrastructure and do watch this webinar to understand the key pain points, which indicate that it’s time to invest in database automation.
AGENDA
DevOps and databases - what are the challenges
Managing databases in a DevOps environment
- Requirements from microservice environments
- Automated deployments
- Performance monitoring
- Backups
- Schema changes
- Version upgrades
- Automated failover
- Integration with ChatOps and other tools
Data distribution
- Database hosting in cloud environments
- Managing data flows
Cloud Automation on AWS
SPEAKERS
Riaan Nolan was the First Puppet Labs Certified Professional in South Africa. Riaan uses Amazon EC2, VPC and Autoscale with Cloudformation to spin up complete stacks with Autoscaling Fleets. He saves companies substantial amounts on their monthly IT bills, typically around 50% - yes, at one company that meant $500k+ per year. And he’s participated in a number of community tech related forums. He uses next generation technologies such as AWS, Cloudformation, Autoscale, Puppet, GlusterFS, NGINX, Magento and PHP to power huge eCommerce stores. His specialties are Puppet Automation, Cloud Deployments, eCommerce, eMarketing, Specialized Linux Services, Windows, Process making, Budgets, Asset Tracking, Procurement.
- Devops Lead, Mukuru
- Expert Live Systems Administrator, foodpanda | Hellofood
- Senior Systems Administrator / Infrastructure Lead, Rocket Internet GmbH
- Senior Technology Manager, Africa Internet Accelerator
Art van Scheppingen is a Senior Support Engineer at Severalnines. He’s a pragmatic MySQL and Database expert with over 15 years experience in web development. He previously worked at Spil Games as Head of Database Engineering, where he kept a broad vision upon the whole database environment: from MySQL to Couchbase, Vertica to Hadoop and from Sphinx Search to SOLR. He regularly presents his work and projects at various conferences (Percona Live, FOSDEM) and related meetups.
Webinar slides: Free Monitoring (on Steroids) for MySQL, MariaDB, PostgreSQL ...Severalnines
Traditional server monitoring tools are not built for modern distributed database architectures. Let’s face it, most production databases today run in some kind of high availability setup - from simpler master-slave replication to multi-master clusters fronted by redundant load balancers. Operations teams deal with dozens, often hundreds of services that make up the database environment.
This is why we built ClusterControl - to address modern, highly distributed database setups based on replication or clustering. We wanted something that could provide a systems view of all the components of a distributed cluster, including load balancers.
Watch this replay of a webinar on free database monitoring using ClusterControl Community Edition. We show you how to monitor all your MySQL, MariaDB, PostgreSQL and MongoDB systems from a single point of control - whether they are deployed as Galera Clusters, sharded clusters or replication setups across on-prem and cloud data centers. We also see how to use Advisors in order to improve performance.
AGENDA
- Requirements for monitoring distributed database systems
- Cloud-based vs On-prem monitoring solutions
- Agent-based vs Agentless monitoring
- Deepdive into ClusterControl Community Edition
- Architecture
- Metrics Collection
- Trending
- Dashboards
- Queries
- Performance Advisors
- Other features available to Community users
SPEAKER
Bartlomiej Oles is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
MongoDB World 2019: Packing Up Your Data and Moving to MongoDB AtlasMongoDB
Moving to a new home is daunting. Packing up all your things, getting a vehicle to move it all, unpacking it, updating your mailing address, and making sure you did not leave anything behind. Well, the move to MongoDB Atlas is similar, but all the logistics are already figured out for you by MongoDB.
Webinar slides: Become a MongoDB DBA - What to Monitor (if you’re really a My...Severalnines
To operate MongoDB efficiently, you need to have insight into database performance. And with that in mind, we’ll dive into monitoring in this second webinar in the ‘Become a MongoDB DBA’ series. MongoDB offers many metrics through various status overviews and commands, but which ones really matter to you? How do you trend and alert on them? What is the meaning behind the metrics?
We’ll discuss the most important ones and describe them in ordinary plain MySQL DBA language. And we’ll have a look at the open source tools available for MongoDB monitoring and trending. Finally, we’ll show you how to leverage ClusterControl’s MongoDB metrics, dashboards, custom alerting and other features to track and optimize the performance of your system.
AGENDA
How does MongoDB monitoring compare to MySQL
Key MongoDB metrics to know about
Trending or alerting?
Available open source MongoDB monitoring tools
How to monitor MongoDB using ClusterControl
Demo
SPEAKER
Art van Scheppingen is a Senior Support Engineer at Severalnines. He’s a pragmatic MySQL and Database expert with over 15 years experience in web development. He previously worked at Spil Games as Head of Database Engineering, where he kept a broad vision upon the whole database environment: from MySQL to Couchbase, Vertica to Hadoop and from Sphinx Search to SOLR. He regularly presents his work and projects at various conferences (Percona Live, FOSDEM) and related meetups.
What if …
- Traditional, labour-intensive backup and archive practices for your MySQL, MariaDB, MongoDB and PostgreSQL databases were a thing of the past?
- You could have one backup management solution for all your business data?
- You could ensure integrity of all your backups?
- You could leverage the competitive pricing and almost limitless capacity of cloud-based backup while meeting cost, manageability, and compliance requirements from the business.
Welcome to our webinar on Backup Management with ClusterControl.
ClusterControl’s centralized backup management for open source databases provides you with hot backups of large datasets, point in time recovery in a couple of clicks, at-rest and in-transit data encryption, data integrity via automatic restore verification, cloud backups (AWS, Google and Azure) for Disaster Recovery, retention policies to ensure compliance, and automated alerts and reporting.
Whether you are looking at rebuilding your existing backup infrastructure, or updating it, this webinar is for you!
AGENDA
- Backup and recovery management of local or remote databases
- Logical or physical backups
- Full or Incremental backups
- Position or time-based Point in Time Recovery (for MySQL and PostgreSQL)
- Upload to the cloud (Amazon S3, Google Cloud Storage, Azure Storage)
- Encryption of backup data
- Compression of backup data
- One centralized backup system for your open source databases (Demo)
- Schedule, manage and operate backups
- Define backup policies, retention, history
- Validation - Automatic restore verification
- Backup reporting
SPEAKER
Bartlomiej Oles, Senior Support Engineer at Severalnines, is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Webinar slides: Designing Open Source Databases for High AvailabilitySeveralnines
It is said that if you are not designing for failure, then you are heading for failure. How do you design a database system from the ground up to withstand failure? This can be a challenge as failures happen in many different ways, sometimes in ways that would be hard to imagine. This is a consequence of the complexity of today’s database environments.
At Severalnines we’re big fans of high availability databases and have seen our fair share of failure scenarios across the thousands of database deployments we enable every year.
In this webinar replay, we’ll look at the different types of failures you might encounter and what mechanisms can be used to address them. We will also look at some of popular HA solutions used today, and how they can help you achieve different levels of availability.
AGENDA
- Why design for High Availability?
- High availability concepts
- CAP theorem
- PACELC theorem
- Trade offs
- Deployment and operational cost
- System complexity
- Performance issues
- Lock management
- Architecting databases for failures
- Capacity planning
- Redundancy
- Load balancing
- Failover and switchover
- Quorum and split brain
- Fencing
- Multi datacenter and multi-cloud setups
- Recovery policy
- High availability solutions
- Database architecture determines Availability
- Active-Standby failover solution with shared storage or DRBD
- Master-slave replication
- Master-master cluster
- Failover and switchover mechanisms
- Reverse proxy
- Caching
- Virtual IP address
- Application connector
SPEAKER
Ashraf Sharif is System Support Engineer at Severalnines. He was previously involved in hosting world and LAMP stack, where he worked as principal consultant and head of support team and delivered clustering solutions for large websites in the South East Asia region. His professional interests are on system scalability and high availability.
Webinar slides: How to Achieve PCI Compliance for MySQL & MariaDB with Cluste...Severalnines
Join Laurent Blume, Unix Systems Engineer & PCI Specialist and Vinay Joosery, CEO at Severalnines, as they discuss all there is to know about how to achieve PCI compliance for MySQL & MariaDB with ClusterControl.
The Payment Card Industry Data Security Standard (PCI-DSS) is a set of technical and operational requirements defined by the PCI Security Standards Council (PCI SSC) to protect cardholder data. These standards apply to all entities that store, process or transmit cardholder data – with requirements for software developers and manufacturers of applications and devices used in those transactions.
PCI data that resides in a MySQL or MariaDB database must of course also adhere to these requirements, and database administrators must follow best practices to ensure the data is secured and compliant. The PCI standards are stringent and can easily require a spiraling amount of time spent on meeting their requirements. Database administrators can end up overwhelmed when using software that was not designed for compliance, often because it long predates PCI itself, as is the case for most database systems in use today.
That is why, as often as possible, reliable tools must be chosen to help with that compliance, easing out the crucial parts. Each time the compliance for one requirement can be shown to be implemented, working, and logged accordingly, time will be saved. If well-designed, it will only require regular software upgrades, a yearly review and a moderate amount of tweaking to follow the standard's evolution over time.
This webinar focuses on PCI-DSS requirements for a MySQL or MariaDB database back-end managed by ClusterControl in order to help meet these requirements. It will provide a MySQL and MariaDB user focussed overview of what the PCI standards mean, how they impact database management and provide valuable tips and tricks on how to achieve PCI compliance for MySQL & MariaDB with ClusterControl.
AGENDA
Introduction to the PCI-DSS standards
The impact of PCI on database management
Step by step review of the PCI requirements
How to meet the requirements for MySQL & MariaDB with ClusterControl
Conclusion
Q&A
This document discusses PostgreSQL database architecture patterns for running PostgreSQL at scale when a relational database as a service like Amazon RDS won't meet needs. It describes challenges faced with MySQL, Redshift and Vertica and how PostgreSQL was better suited through techniques like partitioning by date, TOAST compression, foreign data wrappers, and poor man's parallel processing. Key takeaways are that PostgreSQL supported scaling to petabytes of data, sub-second queries across large date ranges, and custom extensions needed while avoiding limitations and expenses of other database options.
Apache Iceberg - A Table Format for Hige Analytic DatasetsAlluxio, Inc.
Data Orchestration Summit
www.alluxio.io/data-orchestration-summit-2019
November 7, 2019
Apache Iceberg - A Table Format for Hige Analytic Datasets
Speaker:
Ryan Blue, Netflix
For more Alluxio events: https://www.alluxio.io/events/
Join Johan Andersson, CTO at Severalnines, and Ralf Gebhardt, Product Manager at MariaDB, as they unveil the latest release of ClusterControl, the all-inclusive database management system that lets you easily deploy, monitor, manage and scale highly available open source databases - and load balancers - in any environment: on-premise or in the cloud.
We have a particular focus on MariaDB 10.2, thanks to Ralf’s participation, who talk us through the latest features, and give us a sneak preview of what to expect in MariaDB 10.3.
ClusterControl now supports the latest versions of MariaDB, MySQL NDB Cluster and PostgreSQL; and introduces a series of new database backup functionalities that range from AWS & Google Cloud integration backup services to automatic backup verifications, making it ever more efficient to run a solid backup strategy for open source database infrastructures.
We also look at our new operational reports and email notification features - all in a live demo that you don’t want to miss.
AGENDA
- MariaDB 10.2: all the new features and a first look at MariaDB 10.3
- ClusterControl 1.5
- What’s new:
- MariaDB 10.2 support
- AWS & Google Cloud services integration
- Enhanced backup functions
- New features & support for:
- PostgreSQL
- MySQL NDB Cluster
- ProxySQL
- Operational reports
- Live demo
- Q&A
PRESENTERS
Johan Andersson, CTO, Severalnines - Johan's technical background and interest are in high performance computing as demonstrated by the work he did on main-memory clustered databases at Ericsson as well as his research on parallel Java Virtual Machines at Trinity College Dublin in Ireland. Prior to co-founding Severalnines, Johan was Principal Consultant and lead of the MySQL Clustering & High Availability consulting group at MySQL / Sun Microsystems / Oracle, where he designed and implemented large-scale MySQL systems for key customers. Johan is a regular speaker at MySQL User Conferences as well as other high profile community gatherings with popular talks and tutorials around architecting and tuning MySQL Clusters.
Ralf Gebhardt is Product Manager at MariaDB Corporation. He is responsible for MariaDB Server and MariaDB Connectors. He joined MariaDB/SkySQL in 2011 as Principal Sales Engineer.
After 10 years professional experience in Software Development, Support, Training and Consulting, He started working at MySQL GmbH as Sales Engineer in 2002. In the course of the acquisition of Sun Microsystems he joined Oracle, still responsible for MySQL.
He holds a masters degree in Computer Engineering from the University of Cooperative Education (in cooperation with IBM Deutschland).
Workshop on Advanced Design Patterns for Amazon DynamoDB - DAT405 - re:Invent...Amazon Web Services
Join us for the first-ever Amazon DynamoDB practical hands-on workshop. This session is designed for developers, engineers, and database administrators who are involved in designing and maintaining DynamoDB applications. We begin with a walkthrough of proven NoSQL design patterns for at-scale applications. Next, we use step-by-step instructions to apply lessons learned to design DynamoDB tables and indexes that are optimized for performance and cost. Expect to leave this session with the knowledge to build and monitor DynamoDB applications that can grow to any size and scale. Attendees should have a basic understanding of DynamoDB. To attend this workshop, bring your laptop.
Webinar slides: How to Automate & Manage PostgreSQL with ClusterControlSeveralnines
Running PostgreSQL in production comes with the responsibility for a business critical environment; this includes high availability, disaster recovery, and performance. Ops staff worry whether databases are up and running, if backups are taken and tested for integrity, whether there are performance problems that might affect end user experience, if failover will work properly in case of server failure without breaking applications, and the list goes on.
ClusterControl can be used to operationalize your PostgreSQL footprint across your enterprise. It offers a standard way of deploying high-availability replication setups with auto-failover, integrated with load balancers offering a single endpoint to applications. It provides constant health and performance monitoring through rich dashboards, as well as backup management and point-in-time recovery
See how much time and effort can be saved, as well as risks mitigated, with the help of a unified management platform over the more traditional, manual methods.
We’ve seen a 152% increase in ClusterControl installations by PostgreSQL users last year, so make sure you don’t miss out on the trend!
AGENDA
- Managing PostgreSQL “the old way”:
- Common challenges
- Important tasks to perform
- Tools that are available to help
- PostgreSQL automation and management with ClusterControl:
- Deployment
- Backup and recovery
- HA setups
- Failover
- Monitoring
- Live Demo
SPEAKER
Sebastian Insausti, Support Engineer at Severalnines, has loved technology since his childhood, when he did his first computer course (Windows 3.11). And from that moment he was decided on what his profession would be. He has since built up experience with MySQL, PostgreSQL, HAProxy, WAF (ModSecurity), Linux (RedHat, CentOS, OL, Ubuntu server), Monitoring (Nagios), Networking and Virtualization (VMWare, Proxmox, Hyper-V, RHEV).
Prior to joining Severalnines, Sebastian worked as a consultant to state companies in security, database replication and high availability scenarios. He’s also a speaker and has given a few talks locally on InnoDB Cluster and MySQL Enterprise together with an Oracle team. Previous to that, he worked for a Mexican company as chief of sysadmin department as well as for a local ISP (Internet Service Provider), where he managed customers' servers and connectivity.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2lGNybu.
Stefan Krawczyk discusses how his team at StitchFix use the cloud to enable over 80 data scientists to be productive. He also talks about prototyping ideas, algorithms and analyses, how they set up & keep schemas in sync between Hive, Presto, Redshift & Spark and make access easy for their data scientists, etc. Filmed at qconsf.com..
Stefan Krawczyk is Algo Dev Platform Lead at StitchFix, where he’s leading development of the algorithm development platform. He spent formative years at Stanford, LinkedIn, Nextdoor & Idibon, working on everything from growth engineering, product engineering, data engineering, to recommendation systems, NLP, data science and business intelligence.
Csa Summit 2017 - Managing multicloud environmentsCSA Argentina
This document discusses multi-cloud monitoring and analytics using Ceilometer and Gnocchi. It introduces NubeliU's platform for scalable multi-cloud analytics management and monitoring. Gnocchi is presented as Ceilometer's collector and a multi-tenant time-series database for storing large quantities of metrics in an aggregated manner. The document demonstrates how Gnocchi can provide multi-cloud metrics by integrating data from OpenStack, VMware, Azure, AWS and other public clouds.
Bridging Your Business Across the Enterprise and Cloud with MongoDB and NetAppMongoDB
This document discusses how NetApp solutions can help businesses bridge their MongoDB databases across on-premises and cloud environments. It provides an introduction to NetApp and describes how their storage solutions and data fabric can enable hybrid cloud for MongoDB. Specific solutions and technologies discussed include NetApp ONTAP for storage management and provisioning, FlexClone for development/testing, and SolidFire for high performance MongoDB deployments. Customer examples and performance benefits are also summarized.
Similar to Mongo DB Monitoring - Become a MongoDB DBA (20)
DIY DBaaS: A guide to building your own full-featured DBaaSSeveralnines
More so than ever, businesses need to ensure that their databases are resilient, secure, and always available to support their operations. Database-as-a-Service (DBaaS) solutions have become a popular way for organizations to manage their databases efficiently, leveraging cloud infrastructure and advanced set-and-forget automation.
However, consuming DBaaS from providers comes with many compromises. In this guide, we’ll show you how you can build your own flexible DBaaS, your way. We’ll demonstrate how it is possible to get the full spectrum of DBaaS capabilities along with workload access and portability, and avoid surrendering control to a third-party.
From architectural and design considerations to operational requirements, we’ll take you through the process step-by-step, providing all the necessary information and guidance to help you build a DBaaS solution that is tailor-made to your unique use case. So get ready to dive in and learn how to build your own custom DBaaS solution from scratch!
We created this guide to help developers understand:
- Traditional vs. Sovereign DBaaS implementation models
- The DBaaS environment, elements and design principles
- Using a Day 2 operations framework to develop your blueprint
- The 8 key operations that form the foundation of a complete DBaaS
- Bringing the Day 2 ops framework to life with a provisional architecture
- How you can abstract the orchestration layer with Severalnines solutions
Cloud's future runs through Sovereign DBaaSSeveralnines
Sovereign DBaaS is a new way to do DBaaS that allows you to reliably scale your open-source database ops without being limited to a specific environment or ceding control of your infrastructure to third-party service providers.
With Sovereign DBaaS, users can leverage the benefits of modern deployment strategies, e.g. public cloud, hybrid, etc., with additional security, compliance, and risk mitigation. So what exactly is Sovereign DBaaS and why should you choose one?
Presented by Sanjeev Mohan, Principal Analyst at SanjMo and former Gartner Research VP, and Vinay Joosery, CEO of Severalnines, this webinar dives into the future of the cloud and database management and introduces a new solution, Sovereign DBaaS.
The state of the cloud and its current challenges
What is Sovereign DBaaS?
Agenda:
- Key features of Sovereign DBaaS
- Why you should choose a Sovereign DBaaS
- How you can implement Sovereign DBaaS with Severalnines
- Q&A
Working with the Moodle Database: The BasicsSeveralnines
Managing the database behind Moodle is key to improving performance and achieving uptime for your users. In this training video we will talk about the Moodle database including topics like configuration, monitoring, and schema management as well as show you how ClusterControl can help with the management of your eLearning LMS systems.
SysAdmin Working from Home? Tips to Automate MySQL, MariaDB, Postgres & MongoDBSeveralnines
Are you an SysAdmin who is now responsible for your companies database operations? Then this is the webinar for you. Learn from a Senior DBA the basics you need to know to keep things up-and-running and how automation can help.
(slides) Polyglot persistence: utilizing open source databases as a Swiss poc...Severalnines
This document discusses polyglot persistence, which is using multiple specialized databases rather than a single general-purpose database. It provides examples of VidaXL's use of polyglot persistence, including MySQL, MariaDB, PostgreSQL, SOLR, Elasticsearch, MongoDB, Couchbase, and Prometheus. The benefits discussed are using the right database for each job and gaining flexibility as the company transitioned to microservices. Challenges included increased complexity, and solutions involved automation, tooling, and hiring database experts.
Webinar slides: How to Migrate from Oracle DB to MariaDBSeveralnines
This document provides an overview and agenda for a webinar on migrating from Oracle DB to MariaDB. The webinar will cover why organizations are moving to open source databases, the benefits of migrating to MariaDB from Oracle, how to plan and execute the migration process, and post-migration management topics like monitoring, backups, high availability, and scaling in MariaDB. The presentation will include discussions of data type mapping, enabling PL/SQL syntax in MariaDB, available migration tools, and testing approaches.
Webinar slides: How to Manage Replication Failover Processes for MySQL, Maria...Severalnines
Failover is the process of moving to a healthy standby component, during a failure or maintenance event, in order to preserve uptime. The quicker it can be done, the faster you can be back online. However, failover can be tricky for transactional database systems as we strive to preserve data integrity - especially in asynchronous or semi-synchronous topologies. There are risks associated, from diverging datasets to loss of data. Failing over due to incorrect reasoning, e.g., failed heartbeats in the case of network partitioning, can also cause significant harm.
This webinar replay gives a detailed overview of what failover processes may look like in MySQL, MariaDB and PostgreSQL replication setups. We’ve covered the dangers related to the failover process, and discuss the tradeoffs between failover speed and data integrity. We’ve found out about how to shield applications from database failures with the help of proxies. And we've finally had a look at how ClusterControl manages the failover process, and how it can be configured for both assisted and automated failover.
So if you’re looking at minimizing downtime and meet your SLAs through an automated or semi-automated approach, then this webinar replay is for you!
AGENDA
- An introduction to failover - what, when, how
- in MySQL / MariaDB
- in PostgreSQL
- To automate or not to automate
- Understanding the failover process
- Orchestrating failover across the whole HA stack
- Difficult problems
- Network partitioning
- Missed heartbeats
- Split brain
- From assisted to fully automated failover with ClusterControl
- Demo
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
Disaster Recovery Planning for MySQL & MariaDBSeveralnines
Bart Oles - Severalnines AB
Organizations need an appropriate disaster recovery plan to mitigate the impact of downtime. But how much should a business invest? Designing a highly available system comes at a cost, and not all businesses and indeed not all applications need five 9's availability.
We will explain fundamental disaster recovery concepts and walk you through the relevant options from the MySQL & MariaDB ecosystem to meet different tiers of disaster recovery requirements, and demonstrate how to automate an appropriate disaster recovery plan.
Krzysztof Ksiazek - Severalnines AB
So, you are a developer or sysadmin and showed some abilities in dealing with databases issues. And now, you have been elected to the role of DBA. And as you start managing the databases, you wonder…
* How do I tune them to make best use of the hardware?
* How do I optimize the Operating System?
* How do I best configure MySQL or MariaDB for a specific database workload?
If you're asking yourself the following questions when it comes to optimally running your MySQL or MariaDB databases, then this talk is for you!
We will discuss some of the settings that are most often tweaked and which can bring you significant improvement in the performance of your MySQL or MariaDB database. We will also cover some of the variables which are frequently modified even though they should not.
Performance tuning is not easy, especially if you're not an experienced DBA, but you can go a surprisingly long way with a few basic guidelines.
Advanced MySql Data-at-Rest Encryption in Percona ServerSeveralnines
Iwo Panowicz - Percona & Bart Oles - Severalnines AB
The purpose of the talk is to present data-at-rest encryption implementation in Percona Server for MySQL.
Differences between Oracle's MySQL and MariaDB implementation.
- How it is implemented?
- What is encrypted:
- Tablespaces?
- General tablespace?
- Double write buffer/parallel double write buffer?
- Temporary tablespaces? (KEY BLOCKS)
- Binlogs?
- Slow/general/error logs?
- MyISAM? MyRocks? X?
- Performance overhead.
- Backups?
- Transportable tablespaces. Transfer key.
- Plugins
- Keyrings in general
- Key rotation?
- General-Purpose Keyring Key-Management Functions
- Keyring_file
- Is useful? How to make it profitable?
- Keyring Vault
- How does it work?
- How to make a transition from keyring_file
Polyglot Persistence Utilizing Open Source Databases as a Swiss Pocket KnifeSeveralnines
Art Van Scheppingen - vidaXL & Bart Oles - Severalnines AB
Over the past few years, VidaXL has become a European market leader in the online retail of slow moving consumer goods. When a company achieved over 50% year over year growth for the past 9 years, there is hardly enough time to overhaul existing systems. This means existing systems will be stretched to the maximum of their capabilities, and often additional performance will be gained by utilizing a large variety of datastores.
Polyglot persistence reigns in rapidly growing environments and the traditional one-size-fits-all strategy of monoglots is over.
VidaXL has a broad landscape of datastores, ranging from traditional SQL data stores, like MySQL or PostgreSQL alongside more recent load balancing technologies such as ProxySQL, to document stores like MongoDB and search engines such as SOLR and Elasticsearch.
Webinar slides: Our Guide to MySQL & MariaDB Performance TuningSeveralnines
If you’re asking yourself the following questions when it comes to optimally running your MySQL or MariaDB databases:
- How do I tune them to make best use of the hardware?
- How do I optimize the Operating System?
- How do I best configure MySQL or MariaDB for a specific database workload?
Then this replay is for you!
We discuss some of the settings that are most often tweaked and which can bring you significant improvement in the performance of your MySQL or MariaDB database. We also cover some of the variables which are frequently modified even though they should not.
Performance tuning is not easy, especially if you’re not an experienced DBA, but you can go a surprisingly long way with a few basic guidelines.
This webinar builds upon blog posts by Krzysztof from the ‘Become a MySQL DBA’ series.
AGENDA
- What to tune and why?
- Tuning process
- Operating system tuning
- Memory
- I/O performance
- MySQL configuration tuning
- Memory
- I/O performance
- Useful tools
- Do’s and do not’s of MySQL tuning
- Changes in MySQL 8.0
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
Webinar slides: Migrating to Galera Cluster for MySQL and MariaDBSeveralnines
This document provides an overview of online and offline migration strategies for migrating from a standalone MySQL or MySQL master-slave setup to a Galera Cluster. It discusses preparation steps like database schema checks and compatibility. It then outlines the process for offline migration using backups and restore, as well as online migration using MySQL replication to sync data between the existing and new Galera clusters before cutting over. Testing strategies like A/B testing in read-only mode are also presented.
Webinar slides: How to Get Started with Open Source Database ManagementSeveralnines
Join Krzysztof Książek, Senior Support Engineer at Severalnines and expert Database Administrator (DBA), for this 60min crash course on managing open source databases!
If you’ve recently been tasked with taking care of an open source database and are not sure how to proceed; or have worked with open source databases before, but never had to manage them (in other words, you’re not a DBA) … this webinar is for you!
What should you focus on? What are the most important tasks of a DBA?
Krzysztof will share some tips to help you get started on your adventure in open source database management and will try to make it as database-agnostic as possible. In the end, no matter what database you end up managing, some of the tasks and skills required are the same.
So if you’re a sysadmin, DevOps, developer, system architect, … or a DBA in search of a refresher session, make sure to watch this webinar replay!
AGENDA
- What does working with open source databases involve?
- Runbooks
- Testing procedures
- Automation
- Why automate?
- What to automate?
- Tips and tricks for typical procedures in open source database management
- Backups
- High Availability
- Disaster Recovery
- Monitoring
- Health Checks
- Performance Tuning
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxSitimaJohn
Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceIndexBug
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfTechgropse Pvt.Ltd.
In this blog post, we'll delve into the intersection of AI and app development in Saudi Arabia, focusing on the food delivery sector. We'll explore how AI is revolutionizing the way Saudi consumers order food, how restaurants manage their operations, and how delivery partners navigate the bustling streets of cities like Riyadh, Jeddah, and Dammam. Through real-world case studies, we'll showcase how leading Saudi food delivery apps are leveraging AI to redefine convenience, personalization, and efficiency.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Mongo DB Monitoring - Become a MongoDB DBA
1. Copyright 2017 Severalnines AB
MongoDB Monitoring
Art van Scheppingen
Senior Support Engineer, Severalnines
Become a MongoDB DBA - Monitoring Essentials
2. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Monitoring and trending
● Why do we collect data?
● What metrics to collect from MongoDB?
● Key MongoDB metrics in depth
● Available MongoDB monitoring tools
● How to monitor MongoDB using ClusterControl
Agenda
5. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
There is only one person who can land a plane without instruments
6. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Monitoring system (i.e. Nagios)
○ Checks if services are healthy
○ Sends pages
● Trending system (i.e. Cacti, Graphite, Prometheus)
○ Collects metrics
○ Generate graphs
Monitoring vs Trending
7. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Do more than just opening a connection
○ Measure true status of nodes and cluster
○ Test read/write
○ Open essential databases and collections
○ Keep an eye on the replication lag
■ Increase oplog size?
○ Check the full topology
Monitoring: Availability
8. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Trending
○ Plot trends of key (performance) metrics
○ Create timelines of metrics
○ Correlate various metrics
○ Find problems before they arise
○ Pre-emptive problem management
● Trending tools
○ Granularity of sampling
○ More datapoints = better
Trending: why do we need trends?
10. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Periodical (daily/weekly) healthchecks
● Insight into all aspects of the database operations
● Post mortem and proactive monitoring
● Capacity planning
Why do we collect data?
11. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Healthchecks are a pain
● You want to see aggregated
data
● You want to be able to drill
down to a particular host
● You want to see the most
important data first and dig in
later on
Healthchecks
12. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Ability to dig into past data
● Even less than 5s of data
granularity
(hardware-dependent)
● Low granularity allows you to
catch the issue as it evolves -
no need to wait 5 minutes
for a graph to refresh
Post mortem and proactive monitoring
13. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Graphs based on MongoDB
status metrics
● Overall status and per-node
graphs
● Ability to get a timeshifted
graphs - useful for
comparing workload
changes across the time
Insight into internals, capacity planning
15. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Quite similar to other database systems
○ Host metrics
○ Operational metrics
○ Storage engine metrics
○ Replication metrics
○ Shard metrics
Type of metrics to collect
16. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Similar to most other databases
● Understand the utilization of the machine
● Capacity planning
● Determine the type of an issue
○ I/O related?
○ CPU related?
○ Network related?
Host metrics: what for?
17. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● CPU utilization (should I add more nodes to the cluster?)
● Network utilization (am I running out of bandwidth?)
● Ping (how badly latency affects my MongoDB cluster?)
● Disk throughput and IOPS (am I within my hardware limits?)
● Disk space (do I have to plan for larger disks?)
● Memory utilization (do I suffer from a memory leak?)
Host metrics: what to look for?
18. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Similar to most other databases
● Throughput of the cluster
● Relate throughput to cluster performance
● Determine the type of an issue
○ Request spikes?
○ Write amplification related?
○ Queueing?
Operational metrics
19. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Storage engine specific
○ MMAP
○ Wired Tiger
○ MongoRocks
● Insight in how the engine performs
● Internal congestion
Storage engine metrics
20. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Throughput of the replication
● Durability of the oplog
● Replication lag
● Cluster replication acknowledgement
○ Quorum based
○ At least one secondary needs to acknowledge
Replication metrics
22. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Shard chunks and balancing
○ Chunks per shard
○ Disk usage
● Non-sharded collections
○ Sharding has to be enabled on collection level
○ Non-sharded collections get a primary shard assigned
○ Once the primary shard is full, no writes can happen
● Connection pool (mongos)
○ All queries will be sent to the primary in a shard
○ Range queries will block connections of the connection pool
Sharding related metrics
24. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Oplog: a special collection containing all transactions
○ Limited in size (configurable)
○ Eviction of transactions (FIFO)
○ Comparable to a ringbuffer
● Used for replication
○ Secondaries copy transactions from the oplog on other nodes
○ Full data sync necessary once the last executed transaction has been evicted
● Replication window
○ Time between first and last transaction in the oplog
○ Time that allows your secondary to be offline before performing a full sync
Oplog
26. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
From the ClusterControl advisor:
function getReplicationWindow(host) {
var replwindow = {};
replwindow['newset'] = false;
// Fetch the first and last record from the Oplog and take it's timestamp
var res = host.executeMongoQuery("local", '{find: "oplog.rs", sort: { $natural: 1}, limit: 1}');
replwindow['first'] = res["result"]["cursor"]["firstBatch"][0]["ts"]["$timestamp"]["t"];
if (res["result"]["cursor"]["firstBatch"][0]["o"]["msg"] == "initiating set") {
replwindow['newset'] = true;
}
res = host.executeMongoQuery("local", '{find: "oplog.rs", sort: { $natural: -1}, limit: 1}');
replwindow['last'] = res["result"]["cursor"]["firstBatch"][0]["ts"]["$timestamp"]["t"];
replwindow['replwindow'] = replwindow['last'] - replwindow['first'];
return replwindow;
}
Oplog: replication window
27. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● CPU, IO or lock related
● Outcome:
○ Secondary not used by Mongo client drivers
○ Puts larger strain on other secondaries
○ Less likely to be elected during a failover
■ If it will be elected it could be disastrous
○ Lagging behind too far could cause a full sync
Replication lag
29. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Like any other databases: availability
● Client drivers may support connection pooling
○ Multiple non-blocking queries can use the same connection
○ Spawns new connections when low on threshold
● Increase of connections
○ Locking issues
○ Application request bursts
Connections
30. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
From the MongoDB CLI
mongo_replica_0:PRIMARY> db.serverStatus().connections
{ "current" : 25, "available" : 794, "totalCreated" : NumberLong(122418) }
From any mongo client
mongo_replica_0:PRIMARY> db.runCommand( { serverStatus: 1 } ).connections
{ "current" : 25, "available" : 794, "totalCreated" : NumberLong(122418) }
Connections
31. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Atomicity on document level
○ Wiredtiger and MongoRocks
● No “real” transactions
● Write data with the $isolated operator
○ Similar to READ UNCOMMITTED in MySQL (dirty reads in ANSI SQL)
○ No rollback
○ Does not work on shards
Transactions
35. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Optimistic concurrency control
○ If two write operations conflict, the transaction will be paused and retried
● Document level locking
● Tickets (threads)
○ Read
○ Write
Locks (WiredTiger)
37. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● MongoDB uses three tiers of cache
○ Filesystem
○ Active memory
○ Storage engine (WiredTiger / MongoRocks)
● Page faults
○ Cache miss
● Evictions
Cache
38. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
From the MongoDB CLI
mongo_replica_0:PRIMARY> db.serverStatus().extra_info.page_faults
37912924
mongo_replica_0:PRIMARY> db.serverStatus().wiredTiger.cache
{
"bytes currently in the cache" : 887889617,
"modified pages evicted" : 561514,
"tracked dirty pages in the cache" : 626,
"unmodified pages evicted" : 15823118
}
Page faults and cache usage
39. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Shards make write scaling transparently
● Sharding can be solved with two methods:
○ Hash key distribution (limited)
○ Shard lookup table
● MongoDB uses a combination of hash key distribution and shard lookup table
○ Hash key (or range key) distribution gets divided into chunks (ranges)
○ The chunk metadata gets stored in the config server
● The config server is the most important data in a MongoDB sharded cluster!
● The shard router is the the second most important component
● Shards can get out of balance
○ Non-sharded collections
○ Heavy / large writes on a single chunk
○ Auto balancing by the primary of the Config server (3.4) or mongos (< 3.2)
Shard metrics
44. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Open Source
○ Nagios
○ Zabbix
● Subscription based
○ MongoDB Cloud Manager
○ VividCortex
○ ClusterControl
Alerting solutions
45. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Nagios-MongoDB
○ https://github.com/mzupan/nagios-plugin-mongodb/
○ Performs some very important checks
■ Replication lag
■ Lock time percentage
■ Index miss ratio
Nagios
46. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● MongoDB Zabbix monitoring plugin
○ https://github.com/nightw/mikoomi-zabbix-mongodb-monitoring
○ All the necessary metrics and more
■ Entries in oplog
○ Pre-canned triggers
Zabbix
48. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Percona MongoDB Monitoring Templates
○ https://www.percona.com/doc/percona-monitoring-plugins/1.1/cacti/mongodb-templates.
html
Cacti
49. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● PMM
○ https://www.percona.com/doc/percona-monitoring-and-management/
○ Open Source Monitoring & Management framework
○ Can deploy, manage and monitor MySQL & MongoDB
○ Uses Prometheus and Grafana
Orchestration systems: Percona Monitoring & Management
50. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● PMM
○ https://www.percona.com/doc/percona-monitoring-and-management/
○ Open Source Monitoring & Management framework
○ Can deploy, manage and monitor MySQL & MongoDB
○ Uses Prometheus and Grafana
Percona Monitoring & Management sessions:
● MySQL Monitoring with Percona Monitoring and Management, Tue 11:30 - 12:20 in Ballroom E
● Hipster MySQL Monitoring: Serving a deconstructed PMM, Tue 11:30 - 12:20 in Ballroom H
● Monitoring production environment with Percona Monitoring and Management (PMM), Thu 3:00 - 3:50 in room 209
Orchestration systems: Percona Monitoring & Management
59. Copyright 2017 Severalnines ABCopyright 2017 Severalnines AB
● Blog series: Become a MongoDB DBA
○ http://severalnines.com/blog-categories/mongodb
● Webinar series: Become a MongoDB DBA
○ http://severalnines.com/upcoming-webinars
● Visit our website for more resources!
○ http://www.severalnines.com
● Stop by our booth in the exhibit hall
● Other sessions by Severalnines at Percona Live 2017
MySQL Load Balancers - MaxScale, ProxySQL, HAProxy, MySQL Router & nginx - a close up look, Wed
11:10am - 12:00pm in Ballroom D
MySQL (NDB) Cluster Best Practices (Die Hard VIII), Wed 3:30pm - 4:20pm in Room 210
Additional resources