John and I presented the status of Replication in OpenStack Cinder at the Tokyo Summit. This presentation is current for the Liberty release and has some previews of what will be in Mitaka.
Cinder enhancements are proposed to better support replication and other long-running volume operations using stateless snapshots. The enhancements include allowing volume drivers to report capabilities like stateless snapshots, tracking task status separately from volume status, and replicating snapshots between backends. This would enable optimizations like transferring snapshot data directly between storage controllers instead of through Cinder.
Cinder Live Migration and Replication - OpenStack Summit AustinEd Balduf
This document discusses live migration and replication in OpenStack Cinder storage. It describes the process of live migration for instances with block storage, including the different phases from pre-migration to clean-up. It also covers storage compatibility for live migration and the use of config drives. The document then discusses replication in Cinder, including early non-standard implementations and the goals for the "Cheesecake" implementation in Mitaka, which focuses on basic disaster recovery of replicated volumes.
The document describes configuring disaster recovery for OpenStack Cinder volumes using DRBD (Distributed Replicated Block Devices). It discusses setting up DRBD to replicate storage volumes between a primary and secondary node. It then covers configuring LVM over the DRBD devices to create volume groups and logical volumes. Finally, it details the OpenStack Cinder configuration needed to make the replicated volumes available for use by instances.
This document discusses recommended architectures and best practices for deploying Hadoop on VMware vSphere. It recommends deploying Hadoop nodes across multiple virtualization hosts with 10Gb networking for high performance. The standard deployment places data nodes on shared storage and task trackers on local disks. It also discusses planning the cluster size, hardware requirements including CPU, memory, storage and networking considerations. Configuration recommendations include using NTP, proper virtual disk settings, enabling NUMA and avoiding overcommitting resources.
This document discusses optimizing VM images for OpenStack with KVM/QEMU. It covers disk and container formats like RAW, QCOW2, and AMI. It also discusses tools for manipulating disk files, launching an instance, image OS preparation using cloud-init, authentication models, networking configuration, and hotplug support. The goal is to provide optimized images that support features like snapshots while allowing faster instance launching and increased storage efficiency.
XPDS14 - Scaling Xen's Aggregate Storage Performance - Felipe Franciosi, CitrixThe Linux Foundation
This document summarizes Felipe Franciosi's presentation on scaling Xen's aggregate storage performance. It discusses measuring storage performance, the state of the art technologies including grant mapping, persistent grants and tapdisk, and achieving aggregate measurements over 10GB/s using very fast local storage. It also outlines areas for further improvement such as increasing single-VBD performance and enabling many-VBD configurations to perform better by avoiding data copies.
This document compares the disk I/O performance of Xen and KVM virtualization platforms using two types of storage - HDD and SSD. Benchmarking was performed on virtual machines using different I/O sizes and read/write patterns. With HDD storage, Xen generally showed the best performance, while with SSD storage, all VMs exhibited significant delays for small I/Os due to CPU overhead from virtualization. KVM performance was impacted by disk cache settings.
Cinder enhancements are proposed to better support replication and other long-running volume operations using stateless snapshots. The enhancements include allowing volume drivers to report capabilities like stateless snapshots, tracking task status separately from volume status, and replicating snapshots between backends. This would enable optimizations like transferring snapshot data directly between storage controllers instead of through Cinder.
Cinder Live Migration and Replication - OpenStack Summit AustinEd Balduf
This document discusses live migration and replication in OpenStack Cinder storage. It describes the process of live migration for instances with block storage, including the different phases from pre-migration to clean-up. It also covers storage compatibility for live migration and the use of config drives. The document then discusses replication in Cinder, including early non-standard implementations and the goals for the "Cheesecake" implementation in Mitaka, which focuses on basic disaster recovery of replicated volumes.
The document describes configuring disaster recovery for OpenStack Cinder volumes using DRBD (Distributed Replicated Block Devices). It discusses setting up DRBD to replicate storage volumes between a primary and secondary node. It then covers configuring LVM over the DRBD devices to create volume groups and logical volumes. Finally, it details the OpenStack Cinder configuration needed to make the replicated volumes available for use by instances.
This document discusses recommended architectures and best practices for deploying Hadoop on VMware vSphere. It recommends deploying Hadoop nodes across multiple virtualization hosts with 10Gb networking for high performance. The standard deployment places data nodes on shared storage and task trackers on local disks. It also discusses planning the cluster size, hardware requirements including CPU, memory, storage and networking considerations. Configuration recommendations include using NTP, proper virtual disk settings, enabling NUMA and avoiding overcommitting resources.
This document discusses optimizing VM images for OpenStack with KVM/QEMU. It covers disk and container formats like RAW, QCOW2, and AMI. It also discusses tools for manipulating disk files, launching an instance, image OS preparation using cloud-init, authentication models, networking configuration, and hotplug support. The goal is to provide optimized images that support features like snapshots while allowing faster instance launching and increased storage efficiency.
XPDS14 - Scaling Xen's Aggregate Storage Performance - Felipe Franciosi, CitrixThe Linux Foundation
This document summarizes Felipe Franciosi's presentation on scaling Xen's aggregate storage performance. It discusses measuring storage performance, the state of the art technologies including grant mapping, persistent grants and tapdisk, and achieving aggregate measurements over 10GB/s using very fast local storage. It also outlines areas for further improvement such as increasing single-VBD performance and enabling many-VBD configurations to perform better by avoiding data copies.
This document compares the disk I/O performance of Xen and KVM virtualization platforms using two types of storage - HDD and SSD. Benchmarking was performed on virtual machines using different I/O sizes and read/write patterns. With HDD storage, Xen generally showed the best performance, while with SSD storage, all VMs exhibited significant delays for small I/Os due to CPU overhead from virtualization. KVM performance was impacted by disk cache settings.
Kvm performance optimization for ubuntuSim Janghoon
This document discusses various techniques for optimizing KVM performance on Linux systems. It covers CPU and memory optimization through techniques like vCPU pinning, NUMA affinity, transparent huge pages, KSM, and virtio_balloon. For networking, it discusses vhost-net, interrupt handling using MSI/MSI-X, and NAPI. It also covers block device optimization through I/O scheduling, cache mode, and asynchronous I/O. The goal is to provide guidance on configuring these techniques for workloads running in KVM virtual machines.
QEMU Disk IO Which performs Better: Native or threads?Pradeep Kumar
Pradeep Kumar Surisetty from Red Hat presented a comparison of native and threaded I/O performance in QEMU disk I/O. He outlined KVM I/O architecture, storage transport options in KVM including virtio-blk configurations, and benchmark tools used. Performance testing was done with various disk types, file systems, images and configurations. Native generally outperformed threads for random I/O workloads, while threads sometimes showed better performance for sequential reads, especially with multiple VMs.
Konrad Wilk is a Software Development Manager at Oracle. His group’s mission is to make Linux and Xen Project virtualization better and faster. As part of this work, Konrad has been the maintainer of the Xen Project subsystem in Linux, Xen Project maintainer and now also Release Manager for the 4.5 release of the Xen Project Hypervisor. Konrad has been active in the Linux and Xen Project communities for more than 6 years and was instrumental in adding Xen Project support to the Linux Kernel.
TechDay - Toronto 2016 - Hyperconvergence and OpenNebulaOpenNebula Project
Hyperconvergence integrates compute, storage, networking and virtualization resources from scratch in a commodity hardware box supported by a single vendor. It offers scalability, performance, centralized management, reliability and is software-focused. StorPool is a storage software that can be installed on servers to pool and aggregate the capacity and performance of drives. It provides standard block devices and replicates data across drives and servers for redundancy. StorPool integrates fully with Opennebula to provide a robust hyperconverged infrastructure on commodity hardware using distributed storage.
OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBITOpenNebula Project
You will learn what DRBD is, where it came from in its 15 years of existence. How it evolved into a software defined storage solution interesting for users of OpenNebula and why it is very well suited for hyperconverged deployment architectures. The presentation will contain IO performance results and (if time permits) a live demo.
XPDS14 - Intel(r) Virtualization Technology for Directed I/O (VT-d) Posted In...The Linux Foundation
With the development of virtualization, there are more device assignment requirements. Based on VT-d interrupt remapping, Intel introduces VT-d interrupt posting as a more enhanced method to handle interrupts in the virtualization environment. The Posted Interrupts (PI) on CPU side has been already supported in Intel CPUs, with VT-d Posted Interrupt we can get some additional advantages, it can directly deliver external interrupts to running vCPUs without hypervisor involvement, decease the interrupt migration complexity, differentiate between urgent and non-urgent external interrupt, and avoid consuming host-vector for each interrupt to vCPU. In this presentation, Feng will talk about the mechanism of VT-d PI and its advantages, as well as some performance data of I/O intensive workload in Xen, which will show the performance gain after using VT-d PI.
This document summarizes a presentation on KVM optimizations and best practices for both desktop and datacenter use. It covers tools like Libvirtd and virt-manager, virtio drivers, image backends like Qcow2, CPU pinning and cgroups, networking configurations, desktop sharing with SPICE, and challenges in cloud deployments around live migration, storage, and network isolation.
Performance Tuning a Cloud Application: A Real World Case Studyshane_gibson
During the OpenStack Icehouse summit in Atlanta, Symantec presented on our vision for a Key Value as a Service storage technology utilizing MagnetoDB. Since then our Cloud Platform Team has rolled the service out in our production environments. Through that process we have learned about tuning requirements of the solution on bare metal versus hosted VMs within an OpenStack environment.
Our initial performance testing was done with MagnetoDB running on bare metal nodes. After migrating the service from bare metal to an OpenStack VM hosted environment, we observed a 50% reduction in performance.
This presentation will dig into the details of the performance baselines, the tuning of the Nova Compute servers, Virtual Machine settings, and the applications itself to increase our performance.
Why larger community will be interested in this topic
This presentation will dig in to the technical details of performance tuning an application running on an OpenStack Nova Compute cluster. We will examine the performance related configuration settings necessary to improve the hosted application from three different angles:
the underlying compute node Operating System configuration
the hypervisor virtualization layer
and the Guest VM and Application stack
This presentation will provide a real world analysis of the steps taken. In addition, it will provide an outline for other cloud operators to follow when they work towards performance tuning their own cloud stack.
This document discusses various debugging tools and techniques for the Xen hypervisor and guest domains. It describes gdbsx, a gdb server for Xen that allows debugging guest VMs. It also covers the kdb kernel debugger for Xen which can set breakpoints and examine hypervisor state. Other topics include Xen debug keys to trigger crashes and dumps, analyzing core dumps, and Xentrace for capturing trace buffer data.
Ceph Pacific is a major release of the Ceph distributed storage system scheduled for March 2021. It focuses on five key themes: usability, performance, ecosystem integration, multi-site capabilities, and quality. New features in Pacific include automated upgrades, improved dashboard functionality, snapshot-based CephFS mirroring, per-bucket replication in RGW, and expanded telemetry collection. Looking ahead, the Quincy release will focus on continued improvements in these areas such as resource-aware scheduling in cephadm and multi-site monitoring capabilities.
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NECThe Linux Foundation
In recent years Xen has seen the development of many minimalistic or specialized virtual machines (e.g., OSv, Mirage, ClickOS, Erlang on Xen, etc.). Thanks in part to a small CPU and memory footprints, these VMs allow for running thousands or more on a single, inexpensive commodity server. Doing so could save cloud and network operators vast amounts of money.
Attempts to do so are already underway and have discovered important bottlenecks in Xen. While some of these have already been addressed by the community (e.g., limited number of event channels or memory grants) others still remain. In this talk we describe our experience when trying to run up to 10,000 MiniOS-based VMs, including bottlenecks in the XenStore, toolchain and network pipe. We further report on prototypical solutions, and on our implementation of suspend/resume for MiniOS that allows us tens of milliseconds migrations.
Practical advices how to achieve persistence in Redis. Detailed overview of all cons and pros of RDB snapshots and AOF logging. Tips and tricks for proper persistence configuration with Redis pools and master/slave replication.
As eBay is moving to OpenStack, we need to find capacity conversion ratio between ESX and KVM. Moreover, we hope to tunning KVM performance that make KVM to be same as or better than ESX
XPDS16: Xen Live Patching - Updating Xen Without Rebooting - Konrad Wilk, Ora...The Linux Foundation
Oracle and Citrix have been working together to bring live-patching to the Xen hypervisor. This will allow system administrators to update the hypervisor without the need to reboot. The talk will provide an overview of how it works, what were the difficulties in implementing it, how it compares to the other technologies for patching (uSplice, kSplice, kPatch, kGraft, Linux hot-patching), how to use it, and what is in the roadmap schedule.
Practical information on how to Optimize Virtual Machines for High Performance by Boyan Krosnov, Chief Product Officer at StorPool Storage
Presentation delivered at OpenNebula TechDay Sofia on 25-th of February 2016
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...OpenNebula Project
How OpenNebula ease the development and testing of our GNU/Linux distribution?
We are building a turn key GNU/Linux distribution for the Ministère de l’Éducation nationale (France) since 2001 and we start using OpenNebula 3 years ago to smooth the development and test of our solutions. We will follow how our agile team in their day to day use of OpenNebula.
Presentation from 2016 Austin OpenStack Summit.
The Ceph upstream community is declaring CephFS stable for the first time in the recent Jewel release, but that declaration comes with caveats: while we have filesystem repair tools and a horizontally scalable POSIX filesystem, we have default-disabled exciting features like horizontally-scalable metadata servers and snapshots. This talk will present exactly what features you can expect to see, what's blocking the inclusion of other features, and what you as a user can expect and can contribute by deploying or testing CephFS.
Vincent Van der Kussen discusses KVM and related virtualization tools. KVM is a kernel module that allows Linux to function as a hypervisor. It supports x86, PowerPC and s390 architectures. Key tools discussed include libvirt (the virtualization API), virsh (command line tool for libvirt), Qemu (runs virtual machines), and virt-tools like virt-install. The document provides an overview of using these tools to manage virtual machines and storage.
DataStax: Extreme Cassandra Optimization: The SequelDataStax Academy
Al has been using Cassandra since version 0.6 and has spent the last few months doing little else but tune Cassandra clusters. In this talk, Al will show how to tune Cassandra for efficient operation using multiple views into system metrics, including OS stats, GC logs, JMX, and cassandra-stress.
The document announces WordCamp Toronto 2015 on October 3-4 at Humber College Lakeshore Campus. It then provides an overview of managed WordPress hosting by Alex Sirota of NewPath Consulting, including common issues that managed hosting addresses and examples of hosting options like GoDaddy, DreamHost, Flywheel, WP Engine, and Pantheon.
Slides: Introducing the new ClusterControl 1.2.9 - with live demo Severalnines
Highlights of ClusterControl 1.2.9 include:
Support for PostgreSQL Servers
Advanced HAProxy Configurations and Built-in Stats
Hybrid Replication with Galera Clusters
Galera Replication Traffic Encryption
Encrypted Communication between ClusterControl and MySQL-based systems
Query Deadlock Detection in MySQL-based systems
Bootstrap Galera Cluster
Restore of Backups
New UI theme
RPC interface to ClusterControl
Chef Recipe and Puppet Manifest for ClusterControl
Zabbix Plugin for ClusterControl
Kvm performance optimization for ubuntuSim Janghoon
This document discusses various techniques for optimizing KVM performance on Linux systems. It covers CPU and memory optimization through techniques like vCPU pinning, NUMA affinity, transparent huge pages, KSM, and virtio_balloon. For networking, it discusses vhost-net, interrupt handling using MSI/MSI-X, and NAPI. It also covers block device optimization through I/O scheduling, cache mode, and asynchronous I/O. The goal is to provide guidance on configuring these techniques for workloads running in KVM virtual machines.
QEMU Disk IO Which performs Better: Native or threads?Pradeep Kumar
Pradeep Kumar Surisetty from Red Hat presented a comparison of native and threaded I/O performance in QEMU disk I/O. He outlined KVM I/O architecture, storage transport options in KVM including virtio-blk configurations, and benchmark tools used. Performance testing was done with various disk types, file systems, images and configurations. Native generally outperformed threads for random I/O workloads, while threads sometimes showed better performance for sequential reads, especially with multiple VMs.
Konrad Wilk is a Software Development Manager at Oracle. His group’s mission is to make Linux and Xen Project virtualization better and faster. As part of this work, Konrad has been the maintainer of the Xen Project subsystem in Linux, Xen Project maintainer and now also Release Manager for the 4.5 release of the Xen Project Hypervisor. Konrad has been active in the Linux and Xen Project communities for more than 6 years and was instrumental in adding Xen Project support to the Linux Kernel.
TechDay - Toronto 2016 - Hyperconvergence and OpenNebulaOpenNebula Project
Hyperconvergence integrates compute, storage, networking and virtualization resources from scratch in a commodity hardware box supported by a single vendor. It offers scalability, performance, centralized management, reliability and is software-focused. StorPool is a storage software that can be installed on servers to pool and aggregate the capacity and performance of drives. It provides standard block devices and replicates data across drives and servers for redundancy. StorPool integrates fully with Opennebula to provide a robust hyperconverged infrastructure on commodity hardware using distributed storage.
OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBITOpenNebula Project
You will learn what DRBD is, where it came from in its 15 years of existence. How it evolved into a software defined storage solution interesting for users of OpenNebula and why it is very well suited for hyperconverged deployment architectures. The presentation will contain IO performance results and (if time permits) a live demo.
XPDS14 - Intel(r) Virtualization Technology for Directed I/O (VT-d) Posted In...The Linux Foundation
With the development of virtualization, there are more device assignment requirements. Based on VT-d interrupt remapping, Intel introduces VT-d interrupt posting as a more enhanced method to handle interrupts in the virtualization environment. The Posted Interrupts (PI) on CPU side has been already supported in Intel CPUs, with VT-d Posted Interrupt we can get some additional advantages, it can directly deliver external interrupts to running vCPUs without hypervisor involvement, decease the interrupt migration complexity, differentiate between urgent and non-urgent external interrupt, and avoid consuming host-vector for each interrupt to vCPU. In this presentation, Feng will talk about the mechanism of VT-d PI and its advantages, as well as some performance data of I/O intensive workload in Xen, which will show the performance gain after using VT-d PI.
This document summarizes a presentation on KVM optimizations and best practices for both desktop and datacenter use. It covers tools like Libvirtd and virt-manager, virtio drivers, image backends like Qcow2, CPU pinning and cgroups, networking configurations, desktop sharing with SPICE, and challenges in cloud deployments around live migration, storage, and network isolation.
Performance Tuning a Cloud Application: A Real World Case Studyshane_gibson
During the OpenStack Icehouse summit in Atlanta, Symantec presented on our vision for a Key Value as a Service storage technology utilizing MagnetoDB. Since then our Cloud Platform Team has rolled the service out in our production environments. Through that process we have learned about tuning requirements of the solution on bare metal versus hosted VMs within an OpenStack environment.
Our initial performance testing was done with MagnetoDB running on bare metal nodes. After migrating the service from bare metal to an OpenStack VM hosted environment, we observed a 50% reduction in performance.
This presentation will dig into the details of the performance baselines, the tuning of the Nova Compute servers, Virtual Machine settings, and the applications itself to increase our performance.
Why larger community will be interested in this topic
This presentation will dig in to the technical details of performance tuning an application running on an OpenStack Nova Compute cluster. We will examine the performance related configuration settings necessary to improve the hosted application from three different angles:
the underlying compute node Operating System configuration
the hypervisor virtualization layer
and the Guest VM and Application stack
This presentation will provide a real world analysis of the steps taken. In addition, it will provide an outline for other cloud operators to follow when they work towards performance tuning their own cloud stack.
This document discusses various debugging tools and techniques for the Xen hypervisor and guest domains. It describes gdbsx, a gdb server for Xen that allows debugging guest VMs. It also covers the kdb kernel debugger for Xen which can set breakpoints and examine hypervisor state. Other topics include Xen debug keys to trigger crashes and dumps, analyzing core dumps, and Xentrace for capturing trace buffer data.
Ceph Pacific is a major release of the Ceph distributed storage system scheduled for March 2021. It focuses on five key themes: usability, performance, ecosystem integration, multi-site capabilities, and quality. New features in Pacific include automated upgrades, improved dashboard functionality, snapshot-based CephFS mirroring, per-bucket replication in RGW, and expanded telemetry collection. Looking ahead, the Quincy release will focus on continued improvements in these areas such as resource-aware scheduling in cephadm and multi-site monitoring capabilities.
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NECThe Linux Foundation
In recent years Xen has seen the development of many minimalistic or specialized virtual machines (e.g., OSv, Mirage, ClickOS, Erlang on Xen, etc.). Thanks in part to a small CPU and memory footprints, these VMs allow for running thousands or more on a single, inexpensive commodity server. Doing so could save cloud and network operators vast amounts of money.
Attempts to do so are already underway and have discovered important bottlenecks in Xen. While some of these have already been addressed by the community (e.g., limited number of event channels or memory grants) others still remain. In this talk we describe our experience when trying to run up to 10,000 MiniOS-based VMs, including bottlenecks in the XenStore, toolchain and network pipe. We further report on prototypical solutions, and on our implementation of suspend/resume for MiniOS that allows us tens of milliseconds migrations.
Practical advices how to achieve persistence in Redis. Detailed overview of all cons and pros of RDB snapshots and AOF logging. Tips and tricks for proper persistence configuration with Redis pools and master/slave replication.
As eBay is moving to OpenStack, we need to find capacity conversion ratio between ESX and KVM. Moreover, we hope to tunning KVM performance that make KVM to be same as or better than ESX
XPDS16: Xen Live Patching - Updating Xen Without Rebooting - Konrad Wilk, Ora...The Linux Foundation
Oracle and Citrix have been working together to bring live-patching to the Xen hypervisor. This will allow system administrators to update the hypervisor without the need to reboot. The talk will provide an overview of how it works, what were the difficulties in implementing it, how it compares to the other technologies for patching (uSplice, kSplice, kPatch, kGraft, Linux hot-patching), how to use it, and what is in the roadmap schedule.
Practical information on how to Optimize Virtual Machines for High Performance by Boyan Krosnov, Chief Product Officer at StorPool Storage
Presentation delivered at OpenNebula TechDay Sofia on 25-th of February 2016
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...OpenNebula Project
How OpenNebula ease the development and testing of our GNU/Linux distribution?
We are building a turn key GNU/Linux distribution for the Ministère de l’Éducation nationale (France) since 2001 and we start using OpenNebula 3 years ago to smooth the development and test of our solutions. We will follow how our agile team in their day to day use of OpenNebula.
Presentation from 2016 Austin OpenStack Summit.
The Ceph upstream community is declaring CephFS stable for the first time in the recent Jewel release, but that declaration comes with caveats: while we have filesystem repair tools and a horizontally scalable POSIX filesystem, we have default-disabled exciting features like horizontally-scalable metadata servers and snapshots. This talk will present exactly what features you can expect to see, what's blocking the inclusion of other features, and what you as a user can expect and can contribute by deploying or testing CephFS.
Vincent Van der Kussen discusses KVM and related virtualization tools. KVM is a kernel module that allows Linux to function as a hypervisor. It supports x86, PowerPC and s390 architectures. Key tools discussed include libvirt (the virtualization API), virsh (command line tool for libvirt), Qemu (runs virtual machines), and virt-tools like virt-install. The document provides an overview of using these tools to manage virtual machines and storage.
DataStax: Extreme Cassandra Optimization: The SequelDataStax Academy
Al has been using Cassandra since version 0.6 and has spent the last few months doing little else but tune Cassandra clusters. In this talk, Al will show how to tune Cassandra for efficient operation using multiple views into system metrics, including OS stats, GC logs, JMX, and cassandra-stress.
The document announces WordCamp Toronto 2015 on October 3-4 at Humber College Lakeshore Campus. It then provides an overview of managed WordPress hosting by Alex Sirota of NewPath Consulting, including common issues that managed hosting addresses and examples of hosting options like GoDaddy, DreamHost, Flywheel, WP Engine, and Pantheon.
Slides: Introducing the new ClusterControl 1.2.9 - with live demo Severalnines
Highlights of ClusterControl 1.2.9 include:
Support for PostgreSQL Servers
Advanced HAProxy Configurations and Built-in Stats
Hybrid Replication with Galera Clusters
Galera Replication Traffic Encryption
Encrypted Communication between ClusterControl and MySQL-based systems
Query Deadlock Detection in MySQL-based systems
Bootstrap Galera Cluster
Restore of Backups
New UI theme
RPC interface to ClusterControl
Chef Recipe and Puppet Manifest for ClusterControl
Zabbix Plugin for ClusterControl
Fail-Safe Cluster for FirebirdSQL and something moreAlexey Kovyazin
With Firebird HQbird it is possible to create high available cluster or warm standby solution. This presentation defines the problem and describes ways how to create such solutions.
Webinar slides: ClusterControl 1.4: The MySQL Replication & MongoDB Edition -...Severalnines
ClusterControl reduces complexity of managing your database infrastructure while adding support for new technologies; enabling you to truly automate multiple environments for next-level applications. This latest release further builds out the functionality of ClusterControl to allow you to manage and secure your 24/7, mission critical infrastructures.
In this webinar, Johan demonstrated how ClusterControl increases your efficiency by giving you a single interface to deploy and operate your databases, instead of searching for and cobbling together a combination of open source tools, utilities and scripts that need constant updates and maintenance. Watch as ClusterControl demystifies the complexity associated with database high availability, load balancing, recovery and your other everyday struggles.
To put it simply: learn how to be a database hero with ClusterControl!
AGENDA
- ClusterControl (1.4) Overview
- ‘Always on Databases’ with enhanced MySQL Replication functions
- ‘Safer NoSQL’ with MongoDB and larger sharded cluster deployments
- ‘Enabling the DBA’ with ProxySQL, HAProxy and MaxScale
- Backing up your open source databases
- Live Demo
- Q&A
SPEAKER
Johan Andersson, CTO, Severalnines - Johan's technical background and interest are in high performance computing as demonstrated by the work he did on main-memory clustered databases at Ericsson as well as his research on parallel Java Virtual Machines at Trinity College Dublin in Ireland. Prior to co-founding Severalnines, Johan was Principal Consultant and lead of the MySQL Clustering & High Availability consulting group at MySQL / Sun Microsystems / Oracle, where he designed and implemented large-scale MySQL systems for key customers. Johan is a regular speaker at MySQL User Conferences as well as other high profile community gatherings with popular talks and tutorials around architecting and tuning MySQL Clusters.
Git workflows á la-carte, Presenation at jdays2013 www.jdays.se by Nicola Pao...hamidsamadi
This document discusses Git workflows and branching models. It recommends:
1. Using feature branches off a main branch like master for continuous delivery or release branches for product releases.
2. Merging feature branches rather than rebasing to maintain a clear history. Automatic merging is preferred with placeholders to avoid unwanted merges.
3. Setting up tools and automation through hooks for code quality, branch protection and continuous integration while allowing flexibility.
4. Embracing pull requests for collaboration while keeping the main repository structure clean through occasional rebases of feature branches.
Serverless apps: The startup founder's secret weaponArdee Aram
This document discusses serverless apps and computing. It begins by defining serverless computing as a cloud execution model where machine resources are allocated on demand by the cloud provider, removing the need for customers to manage servers themselves. It then outlines the benefits of the serverless model for student founders, including low costs since apps are only charged when in use (as opposed to monthly server fees) and the ability to easily try out projects without ongoing hosting fees. Various serverless technologies like AWS Lambda and Netlify Functions are also introduced.
This document discusses automating development operations and introduces seven capabilities for effective DevOps: isolation, portability, automated configuration, simplified code reviews, continuous integration and testing, atomic deployments, and automated monitoring. For each capability, it outlines challenges and lists common automation options and tools to address those challenges, noting pros and cons of different approaches. It emphasizes that automation should adapt to existing workflows, scale gradually, and not reduce developer productivity.
Moving faster with CI/CD: Best DevOps practices and lessons learntMalinda Kapuruge
The document discusses implementing continuous integration (CI) and continuous delivery (CD) practices to reduce time to market and quickly fix production issues. It describes setting up automated testing, code scanning, and infrastructure as code in a CI pipeline using tools like GitHub Actions, Jest, Docker, and Terraform. This allows small code changes to be frequently committed, tested, and merged while avoiding large deployments. Changes are then continuously delivered to staged environments like development, test, and production through automated deployment processes.
This document provides an overview and summary of key points from a presentation on designing virtual infrastructures and hypervisors. It discusses pre-requisites, assessing which servers are good candidates for virtualization, measuring server performance, determining the right amount of RAM for virtual machines, different types of virtualization technologies, high availability options, and live migration capabilities.
IBM Connect 2017: Back from the Dead: When Bad Code Kills a Good ServerSerdar Basegmez
The document provides a case study describing performance issues experienced by a critical Domino server at a company, referred to as Acme, Inc. It details the steps taken to analyze the problem, which included collecting log and system usage data, inspecting Domino and application configuration files, and isolating potential causes. The analysis revealed that agents were running at inappropriate times and overlapping with other maintenance tasks, negatively impacting database performance. Code in a critical application was also identified as inefficiently performing large searches. The document outlines how the operating system, Domino, and application configurations were optimized to address these issues and ensure tasks had adequate resources. The results included significantly improved system activity index, database usage and compression, and resolution of odd overnight
Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...Western Digital
By now, Tier 1 apps deployed on flash is ubiquitous. However, Tier 2 apps often remain relegated to spinning media. This presentation explains the economics of consolidating on flash. Owing to the SQL Server core licensing model, licensing a 2-socket commodity server can cost up to $500,000 or more! Consolidating instances on flash can—and does—save hundreds of thousands to millions of dollars. This presentation provides real-life case studies showing such real-life savings.
Developing Rails Apps in Technical IsolationJesus Jackson
This document discusses techniques for developing Ruby on Rails applications in a restricted technical environment where internet access is limited. It addresses problems related to managing Ruby versions and gems, deployment with Capistrano, testing email, using Git with a Subversion-based workflow, and hosting gems. Specific solutions covered include RVM::FW for managing Ruby versions behind a firewall, MailCatcher for testing email locally, using git-svn to work with Git and Subversion, and Gem In A Box or a local gem server for hosting gems. The document provides detailed instructions for setting up and using each of these solutions.
Locking down your Kubernetes cluster with LinkerdBuoyant
In this hands-on workshop, we cover the basics of locking down in-cluster network traffic using the new traffic policies introduced in Linkerd 2.11. Using Linkerd’s ability to authorize traffic based on workload identity, we cover a variety of practical use cases, including restricting access to a critical service, preventing traffic across namespaces, and locking down traffic while still allowing metrics scrapes, health checks, and other meta-traffic.
Denodo Solution Manager: Best Practices for PromotionDenodo
Watch full webinar here: https://bit.ly/3bk5vYD
What is the best way to manage metadata promotion? How do organizations seamlessly promote the metadata, users, and their permissions? How can administrators guarantee the stability of the service in the event of failure?
Join us for this session with María Sordo, Technical Account Manager, Denodo, and get key insights on how to promote the metadata using Solution Manager and walk away with best practices and practical tips.
Watch On-Demand and Learn:
- Recommended steps through the promotion process using Solution Manager
- Permissions management during the promotion
- Backup and rollback considerations
- Challenges with the promotion and practical tips for overcoming them
EuroPython 2011 - How to build complex web applications having fun?Andrew Mleczko
Web development is a complexity challenge nowadays. Growing number of functionalities results in customer expectations increase which makes project design more difficult. Using proper tools that suite your customer needs is essential.
This talk is about successful story using closely together Pyramid and Plone. Basing on these examples you will see the main reasons for using Plone as a CMS only and letting Pyramid do the rest (vertical application).
Slide Deck from our 2013 SANDCamp presentation. More of the content was likely captured in the conversation, as we used this deck as a jumping off point for the chat, but there's still some worthwhile concepts in there.
This document summarizes 7 tools for a DevOps stack: Puppet, MCollective, Logstash, Kibana, Graphite, Vagrant, and FPM. It provides an overview of what each tool is used for, such as configuration management with Puppet, distributed orchestration with MCollective, log aggregation with Logstash, log visualization with Kibana, metrics collection and graphing with Graphite, infrastructure automation with Vagrant, and packaging with FPM. The document also includes examples and screenshots demonstrating how some of the tools can be used.
Veeam Webinar - Case study: building bi-directional DRJoep Piscaer
This document outlines a case study for building bidirectional disaster recovery (DR) between two virtualized infrastructures located on separate sites. The project goals were to reduce recovery time objectives (RTO) from weeks to hours, reduce recovery point objectives (RPO) from infinite to a day, and implement a DR solution using Veeam software. The solution involved using Veeam's distributed backup architecture with proxies and repositories on each site to back up VMs locally and to the remote site. Reverse incremental backups were used to minimize storage usage. A live demo was presented to showcase the solution.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Webinar: Designing a schema for a Data WarehouseFederico Razzoli
Are you new to data warehouses (DWH)? Do you need to check whether your data warehouse follows the best practices for a good design? In both cases, this webinar is for you.
A data warehouse is a central relational database that contains all measurements about a business or an organisation. This data comes from a variety of heterogeneous data sources, which includes databases of any type that back the applications used by the company, data files exported by some applications, or APIs provided by internal or external services.
But designing a data warehouse correctly is a hard task, which requires gathering information about the business processes that need to be analysed in the first place. These processes must be translated into so-called star schemas, which means, denormalised databases where each table represents a dimension or facts.
We will discuss these topics:
- How to gather information about a business;
- Understanding dictionaries and how to identify business entities;
- Dimensions and facts;
- Setting a table granularity;
- Types of facts;
- Types of dimensions;
- Snowflakes and how to avoid them;
- Expanding existing dimensions and facts.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
Project Management Semester Long Project - Acuityjpupo2018
Acuity is an innovative learning app designed to transform the way you engage with knowledge. Powered by AI technology, Acuity takes complex topics and distills them into concise, interactive summaries that are easy to read & understand. Whether you're exploring the depths of quantum mechanics or seeking insight into historical events, Acuity provides the key information you need without the burden of lengthy texts.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Choosing The Best AWS Service For Your Website + API.pptx
Cinder - status of replication
1. The status of Replication
John Griffith - john.griffith@solidfire.com
Ed Balduf - ed.balduf@solidfire.com
2. Why are we building replication into Cinder?
▪ What’s it for? What does it buy me?
▪ It’s a DR strategy for non-cloud applications.
▪ Backup (Kind of).
▪ Limited use cases today.
▪ Different vendors
▪ Different terminology for states.
▪ How does failover/failback work?
▪ Crawl-walk-run
▪ Should be aware of Availability Groups (future).
3. Use Cases Considered
▪ NEVER - 2 different vendor backends.
▪ Consider:
▪ 2 backends, same cloud
▪ 2 backends, different cloud
▪ 2 backends, one not in a cloud
▪ Replication one to multiple backends
▪ Automated -vs- Non-Automated failover
▪ Snapshot replication -vs - Continuous replication
4. How we got here
Missteps
▪ Many voices
▪ Lack of agreed upon vision
▪ Difference in interpretation
▪ Unique characteristics that don’t translate well
▪ Lack of testability
▪ Rush at the end of release cycle
5. V1 lessons learned
▪ Lack of community involvement
▪ Only worked for one vendor
▪ Not clearly understood
▪ Documentation was not up to standards
6. V2 Learning from our mistakes
▪ Heavy involvement from multiple vendors
▪ Reviews, reviews, reviews
▪ DON’T RUSH!!!
▪ We will sell no wine, before it’s time
7. How it works and what it does:
Driver must report:
replication_enabled = True
In it’s capabilities.
New config variable:
replication_device = {
device_target_id: <required>,
managed_backend_name: <host@backend_name>,
Vendor Key1: <Vendor Value>,
Vendor Key2: <Vendor Value>,
}
(Note: No trailing comma allowed in replication_device KV pair list)