This document discusses best practices for high availability (HA) and replication of PostgreSQL databases in virtualized environments. It covers enterprise needs for HA, technologies like VMware HA and replication that can provide HA, and deployment blueprints for HA, read scaling, and disaster recovery within and across datacenters. The document also discusses PostgreSQL's different replication modes and how they can be used for HA, read scaling, and disaster recovery.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
Advance Sharding Solution with ProxySQL
ProxySQL is a very powerful platform that allows us to manipulate and manage our connections and queries in a simple but effective way.
Historically MySQL lacks in sharding capability. This significant missing part had often cause developer do implement sharding at application level, or DBA/SA to move on to another solution.
ProxySQL comes with an elegant and simple solution that allow us to implement sharding capability with MySQL without the need to perform significant, or at all, changes in the code.
This brief presentation will illustrate how to successfully configure and use ProxySQL to perform sharding, from very simple approach based on connection user/ip/port, to complicate ones that see the need to read values inside queries.
사례로 알아보는 MariaDB 마이그레이션
현대적인 IT 환경과 애플리케이션을 만들기 위해 우리는 오늘도 고민을 거듭합니다. 최근 들어 오픈소스 DB가 많은 업무에 적용되고 검증이 되면서, 점차 무거운 상용 데이터베이스를 가벼운 오픈소스 DB로 전환하는 움직임이 대기업의 미션 크리티컬 업무까지로 확산하고 있습니다. 이는 클라우드 환경 및 마이크로 서비스 개념 확산과도 일치하는 움직임입니다.
상용 DB를 MariaDB로 이관한 사례를 통해 마이그레이션의 과정과 효과를 살펴 볼 수 있습니다.
MariaDB로 이관하는 것은 어렵다는 생각을 막연히 가지고 계셨다면 본 자료를 통해 이기종 데이터베이스를 MariaDB로 마이그레이션 하는 작업이 어렵지 않게 수행될 수 있다는 점을 실제 사례를 통해 확인하시길 바랍니다.
웨비나 동영상
https://www.youtube.com/watch?v=xRsETZ5cKz8&t=52s
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
Advance Sharding Solution with ProxySQL
ProxySQL is a very powerful platform that allows us to manipulate and manage our connections and queries in a simple but effective way.
Historically MySQL lacks in sharding capability. This significant missing part had often cause developer do implement sharding at application level, or DBA/SA to move on to another solution.
ProxySQL comes with an elegant and simple solution that allow us to implement sharding capability with MySQL without the need to perform significant, or at all, changes in the code.
This brief presentation will illustrate how to successfully configure and use ProxySQL to perform sharding, from very simple approach based on connection user/ip/port, to complicate ones that see the need to read values inside queries.
사례로 알아보는 MariaDB 마이그레이션
현대적인 IT 환경과 애플리케이션을 만들기 위해 우리는 오늘도 고민을 거듭합니다. 최근 들어 오픈소스 DB가 많은 업무에 적용되고 검증이 되면서, 점차 무거운 상용 데이터베이스를 가벼운 오픈소스 DB로 전환하는 움직임이 대기업의 미션 크리티컬 업무까지로 확산하고 있습니다. 이는 클라우드 환경 및 마이크로 서비스 개념 확산과도 일치하는 움직임입니다.
상용 DB를 MariaDB로 이관한 사례를 통해 마이그레이션의 과정과 효과를 살펴 볼 수 있습니다.
MariaDB로 이관하는 것은 어렵다는 생각을 막연히 가지고 계셨다면 본 자료를 통해 이기종 데이터베이스를 MariaDB로 마이그레이션 하는 작업이 어렵지 않게 수행될 수 있다는 점을 실제 사례를 통해 확인하시길 바랍니다.
웨비나 동영상
https://www.youtube.com/watch?v=xRsETZ5cKz8&t=52s
MySQL Group Replication - HandsOn TutorialKenny Gryp
During this tutorial, attendees have their hands on virtual machines and migrate standard Master - Slave architecture to the new MySQL native Group Replication.
After explaining briefly what is group replication and how this is important for MySQL HA architecture. We will cover how to verify the workload and the scheme to how GR can be used and configured.
Then we will go trough the migration steps with minimal impact on the live system.
Basic administration tasks are covered such as add/remove a node from the cluster. We also play with performance_schema to monitor our Group Replication cluster and understand how to control it.
This ppt was used by Devrim at pgDay Asia 2017. He talked about some important facts about WAL - Transaction Logs or xlogs in PostgreSQL. Some of these can really come handy on a bad day
A look at what HA is and what PostgreSQL has to offer for building an open source HA solution. Covers various aspects in terms of Recovery Point Objective and Recovery Time Objective. Includes backup and restore, PITR (point in time recovery) and streaming replication concepts.
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
Following MySQL InnoDB Cluster as our first, fully integrated MySQL High Availability solution based on Group Replication, MySQL Shell 8.0.19 includes MySQL InnoDB ReplicaSet which delivers another complete solution, this time based on MySQL Replication.
The basic idea for InnoDB ReplicaSet is to do the same for classic MySQL Replication as InnoDB Cluster did for Group Replication. We take a strong technology that is very powerful but can be complex, and provide an easy-to-use AdminAPI for it in the MySQL Shell.
In just a few easy to use Shell commands, a MySQL Replication database architecture can be configured from scratch including:
Data provisioning using MySQL CLONE, Setting up replication,
Performing manual switchover/failover.
Apache Spark Data Source V2 with Wenchen Fan and Gengliang WangDatabricks
As a general computing engine, Spark can process data from various data management/storage systems, including HDFS, Hive, Cassandra and Kafka. For flexibility and high throughput, Spark defines the Data Source API, which is an abstraction of the storage layer. The Data Source API has two requirements.
1) Generality: support reading/writing most data management/storage systems.
2) Flexibility: customize and optimize the read and write paths for different systems based on their capabilities.
Data Source API V2 is one of the most important features coming with Spark 2.3. This talk will dive into the design and implementation of Data Source API V2, with comparison to the Data Source API V1. We also demonstrate how to implement a file-based data source using the Data Source API V2 for showing its generality and flexibility.
MySQL Administrator
Basic course
- MySQL 개요
- MySQL 설치 / 설정
- MySQL 아키텍처 - MySQL 스토리지 엔진
- MySQL 관리
- MySQL 백업 / 복구
- MySQL 모니터링
Advanced course
- MySQL Optimization
- MariaDB / Percona
- MySQL HA (High Availability)
- MySQL troubleshooting
네오클로바
http://neoclova.co.kr/
PostgreSQL 10 had added built-in logical replication which tackles some of the limitations of physical replication and opens up the possibility of promising new areas of replication. In this webinar, we will introduce the concept of logical replication and demonstrate how one can configure a logical replication in no time.
Highlights include:
- Basic architecture, including the publisher and subscriber model
- Configuration, administration, and monitoring
- Limitations and future plans
Architecture for building scalable and highly available Postgres ClusterAshnikbiz
As PostgreSQL has made way into business critical applications, many customers who are using Oracle RAC for high availability and load balancing have asked for similar functionality for using PostgreSQL.
In this Hangout session we would discuss architecture and alternatives, based on real life experience, for achieving high availability and load balancing functionality when you deploy PostgreSQL. We will also present some of the key tools and how to deploy them for effectiveness of this architecture.
MySQL Group Replication - HandsOn TutorialKenny Gryp
During this tutorial, attendees have their hands on virtual machines and migrate standard Master - Slave architecture to the new MySQL native Group Replication.
After explaining briefly what is group replication and how this is important for MySQL HA architecture. We will cover how to verify the workload and the scheme to how GR can be used and configured.
Then we will go trough the migration steps with minimal impact on the live system.
Basic administration tasks are covered such as add/remove a node from the cluster. We also play with performance_schema to monitor our Group Replication cluster and understand how to control it.
This ppt was used by Devrim at pgDay Asia 2017. He talked about some important facts about WAL - Transaction Logs or xlogs in PostgreSQL. Some of these can really come handy on a bad day
A look at what HA is and what PostgreSQL has to offer for building an open source HA solution. Covers various aspects in terms of Recovery Point Objective and Recovery Time Objective. Includes backup and restore, PITR (point in time recovery) and streaming replication concepts.
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
Following MySQL InnoDB Cluster as our first, fully integrated MySQL High Availability solution based on Group Replication, MySQL Shell 8.0.19 includes MySQL InnoDB ReplicaSet which delivers another complete solution, this time based on MySQL Replication.
The basic idea for InnoDB ReplicaSet is to do the same for classic MySQL Replication as InnoDB Cluster did for Group Replication. We take a strong technology that is very powerful but can be complex, and provide an easy-to-use AdminAPI for it in the MySQL Shell.
In just a few easy to use Shell commands, a MySQL Replication database architecture can be configured from scratch including:
Data provisioning using MySQL CLONE, Setting up replication,
Performing manual switchover/failover.
Apache Spark Data Source V2 with Wenchen Fan and Gengliang WangDatabricks
As a general computing engine, Spark can process data from various data management/storage systems, including HDFS, Hive, Cassandra and Kafka. For flexibility and high throughput, Spark defines the Data Source API, which is an abstraction of the storage layer. The Data Source API has two requirements.
1) Generality: support reading/writing most data management/storage systems.
2) Flexibility: customize and optimize the read and write paths for different systems based on their capabilities.
Data Source API V2 is one of the most important features coming with Spark 2.3. This talk will dive into the design and implementation of Data Source API V2, with comparison to the Data Source API V1. We also demonstrate how to implement a file-based data source using the Data Source API V2 for showing its generality and flexibility.
MySQL Administrator
Basic course
- MySQL 개요
- MySQL 설치 / 설정
- MySQL 아키텍처 - MySQL 스토리지 엔진
- MySQL 관리
- MySQL 백업 / 복구
- MySQL 모니터링
Advanced course
- MySQL Optimization
- MariaDB / Percona
- MySQL HA (High Availability)
- MySQL troubleshooting
네오클로바
http://neoclova.co.kr/
PostgreSQL 10 had added built-in logical replication which tackles some of the limitations of physical replication and opens up the possibility of promising new areas of replication. In this webinar, we will introduce the concept of logical replication and demonstrate how one can configure a logical replication in no time.
Highlights include:
- Basic architecture, including the publisher and subscriber model
- Configuration, administration, and monitoring
- Limitations and future plans
Architecture for building scalable and highly available Postgres ClusterAshnikbiz
As PostgreSQL has made way into business critical applications, many customers who are using Oracle RAC for high availability and load balancing have asked for similar functionality for using PostgreSQL.
In this Hangout session we would discuss architecture and alternatives, based on real life experience, for achieving high availability and load balancing functionality when you deploy PostgreSQL. We will also present some of the key tools and how to deploy them for effectiveness of this architecture.
There are many ways to run high availability with PostgreSQL. Here, we present a template for you to create your own customized, high-availability solution using Python and for maximum accessibility, a distributed configuration store like ZooKeeper or etcd.
Josh Berkus
You've heard that PostgreSQL is the highest-performance transactional open source database, but you're not seeing it on YOUR server. In fact, your PostgreSQL application is kind of poky. What should you do? While doing advanced performance engineering for really high-end systems takes years to learn, you can learn the basics to solve performance issues for 80% of PostgreSQL installations in less than an hour. In this session, you will learn: -- The parts of database application performance -- The performance setup procedure -- Basic troubleshooting tools -- The 13 postgresql.conf settings you need to know -- Where to look for more information.
This presentation surveys different ways one can geographically distribute PostgreSQL, including master-slave and multi-master solutions. It discusses pitfalls and emphasizes understanding requirements. The presentation covers some of the existing tools that are available in the community. It also touches upon upcoming PostgreSQL solutions.
This presentation explores a broad cross-section of enterprise Postgres deployments to identify key usage patterns and reveals important aspects of performance, scalability, and availability including:
* Challenges organizations encounter most frequently during the stages of database development, deployment and maintenance
* Tuning parameters used most frequently to improve performance of production databases
* Frequently problematic database maintenance processes and configuration parameters
* Most commonly-used database back-up and recovery strategies
Best Practices for a Complete Postgres Enterprise Architecture SetupEDB
This presentation provides the details of a best-practice reference architecture for deploying Postgres into your enterprise for large scale OLTP solutions. It reviews how to put all the key pieces together to build a robust, reliable and cost-effective Postgres infrastructure, providing recommendations for configuration and deployment guidance.
This presentation reviews:
* Standard requirements for robust and reliable OTLP architecture
* How to use open source based Postgres Plus building blocks to meet those requirements
* High availability system design with streaming replication
* Backup with logical and physical backup recommendations and setup for point-in-time recovery
* Replication – single master and multi-master considerations
* Database infrastructure monitoring with alerts
* Managing and tuning your Postgres database configuration
To listen to the recording visit www.enterprisedb.com - Resources - Webcasts - On Demand webcasts
Email sales@enterprisedb.com with your questions about Postgres.
Best Practices for Becoming an Exceptional Postgres DBA EDB
Drawing from our teams who support hundreds of Postgres instances and production database systems for customers worldwide, this presentation provides real-real best practices from the nation's top DBAs. Learn top-notch monitoring and maintenance practices, get resource planning advice that can help prevent, resolve, or eliminate common issues, learning top database tuning tricks for increasing system performance and ultimately, gain greater insight into how to improve your effectiveness as a DBA.
Supersized PostgreSQL: Postgres-XL for Scale-Out OLTP and Big Data Analyticsmason_s
In this talk we introduce Postgres-XL for scaling out PostgreSQL. We cover its architecture, how tables are distributed, and include a sample configuration for a small local test cluster. Finally, we discuss the differences to PostgreSQL and discuss Postgres-XL community building
This presentation answer a lot of your questions about PostgreSQL and the Red Hat Cluster Suite.
It reviews how you can create failover/standby capabilities with the following activities:
General PostgreSQL clustering options
Overview of Red Hat Cluster Service
Identification of candidate databases for clustering
Identification of hardware for clustering
Analysis of uptime requirements and data latency
Implementation of clustering
Testing of clustering
PostgreSQL installation tips for RHCS
This presentation covers all aspects of PostgreSQL administration, including installation, security, file structure, configuration, reporting, backup, daily maintenance, monitoring activity, disk space computations, and disaster recovery. It shows how to control host connectivity, configure the server, find the query being run by each session, and find the disk space used by each database.
The paperback version is available on lulu.com there http://goo.gl/fraa8o
This is the first volume of the postgresql database administration book. The book covers the steps for installing, configuring and administering a PostgreSQL 9.3 on Linux debian. The book covers the logical and physical aspect of PostgreSQL. Two chapters are dedicated to the backup/restore topic.
Slides from Secon'2015 - Software Developers Conference. Penza, Russia.
The database is an essential element of any project. The database must be stable and provide good performance. If you plan to use PostgreSQL in your project, you will run into question the choice of operating system. Linux is one of the most popular operating system today. The combination of flexibility and stability makes Linux a good candidate as a platform for PostgreSQL. However, the default settings are suitable for a wide range of workloads. In this report, I will talk about what settings should pay attention and how they affect the performance of PostgreSQL. Which of these settings are more important, and what - no. How do the PostgreSQL more predictable and stable under normal circumstances or in cases of increasing load.
Introduction to failover clustering with sql serverEduardo Castro
In this presentation we review the basic requirements to install a SQL Server Failover Cluster.
Regards,
Eduardo Castro Martinez
http://ecastrom.blogspot.com
http://comunidadwindows.org
Enterprises routinely build Disaster Recovery and High Availability measures into their private cloud environments. So why do downtime and data loss risks still exist?
VMworld 2013: Implementing a Holistic BC/DR Strategy with VMware - Part TwoVMworld
VMworld 2013
Jeff Hunter, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
Ken Werneburg, VMware
Implementing a Disaster Recovery Solution using VMware Site Recovery Manager ...Paula Koziol
IBM Spectrum Virtualize delivers business continuity capabilities using a stretched cluster configuration together with VMware Site Recovery Manager (SRM). The result is an end-to-end disaster recovery solution for organizations of all sizes. Join this session to understand how IBM Spectrum Virtualize, including offerings like IBM SAN Volume Controller (SVC) and IBM Storwize Family, integrates with VMware SRM to automate and optimize disaster recovery operations. Everyone who works in mission critical environments understands the need for high availability and effective solutions for planned and unplanned outages. Organizations demand disaster recovery operations that are fully automated and can be executed in a repeatable manner, so that they are always prepared for disaster situations. This IBM-VMware solution offers SMB and enterprise customers the ability to survive a wide range of failures and enables seamless migration of applications across company sites for various planned activities, enabling zero-downtime application mobility.
Lessons from the field: Catalog of Kafka Deployments | Joseph Niemiec, ClouderaHostedbyConfluent
Streaming architectures have been on the rise steadily and as a result, we have seen the adoption of Kafka go up too. With the diverse spread of use cases across multiple industries, we have seen a variety of Kafka deployments across our hundreds of Kafka customers. Along the way, we have learnt some best practices as well as what not to do in mission-critical architectures. Join Joe Niemiec, Sr. Product Manager at Cloudera, as he shares these insights in this session that covers topics such as - The many ways that Kafka has been deployed in the field Standalone clusters, multiple clusters in a single data center and multiple clusters geographically distributed performing replication Clusters of all sizes small and large, few messages to hundreds of thousands per second Discussion about architecture failure domains Configurations tuned and used in specific deployments
What is coming for VMware vSphere?
Delivered at VMUG DK/UK/BE in November 2014. Session is all about vSphere futures, what can be expected in the near future.
VMworld 2013: Three Advantages of Running Cloud Foundry in a VMware Private C...VMworld
VMworld 2013
Tarik Dwiek, EMC
Steve Flanders, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
VMworld 2013: Maximize Database Performance in Your Software-Defined Data CenterVMworld
VMworld 2013
Mark Achtemichuk, VMware
Michael Webster, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
If you need to build highly performant, mission critical ,microservice-based system following DevOps best practices, you should definitely check Service Fabric!
Service Fabric is one of the most interesting services Azure offers today. It provide unique capabilities outperforming competitor products.
We are seeing global companies start to use Service Fabric for their mission critical solutions.
In this talk we explore the current state of Service Fabric and dive deeper to highlight best practices and design patterns.
We will cover the following topics:
• Service Fabric Core Concepts
• Cluster Planning and Management
• Stateless Services
• Stateful Services
• Actor Model
• Availability and reliability
• Scalability and perfromance
• Diganostics and Monitoring
• Containers
• Testing
• IoT
Live broadcast on https://www.youtube.com/watch?v=Zuxfhpab6xo
The have no fear guide to virtualizing databasesSolarWinds
When it comes to a successful database virtualization journey, there are things you must know before you start. In this presentation you will:
-Review terms and concepts for VMware, by far the most common virtualization platform
-Examine how to use vSphere (the VMware admin console)
-Explore the differences between virtual and physical host metrics a
-Learn to overcome the shortcomings of virtualizing your database environment
PostgreSQL is designed to be easily extensible. For this reason, extensions loaded into the database can function just like features that are built in. In this session, we will learn more about PostgreSQL extension framework, how are they built, look at some popular extensions, management of these extensions in your deployments.
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
In this session we will deep dive into the exciting features of Amazon RDS for PostgreSQL, including new versions of PostgreSQL releases, new extensions, larger instances. We will also show benchmarks of new RDS instance types, and their value proposition. We will also look at how high availability and read scaling works on RDS PostgreSQL. We will also explore lessons we have learned managing a large fleet of PostgreSQL instances, including important tunables.
My experience with embedding PostgreSQLJignesh Shah
At my current company, we embed PostgreSQL based technologies in various applications shipped as shrink-wrapped software. In this session we talk about the experience of embedding PostgreSQL where it is not directly exposed to end-user and the issues encountered on how they were resolved.
We will talk about business reasons,technical architecture of deployments, upgrades, security processes on how to work with embedded PostgreSQL databases.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
2. Agenda
§ Enterprise needs
§ Technologies readily available
• Virtualization Technologies for HA
• Replication modes of PostgreSQL (in core)
§ Datacenter Deployment Blueprints
• HA within Datacenter,
• Read-Scaling within Datacenter
• DR/Read-Scaling across Datacenters
2
4. Causes of Downtime
§ Planned Downtime
• Software upgrade (OS patches, SQL Server cumulative updates)
• Hardware/BIOS upgrade
§ Unplanned Downtime
• Datacenter failure (natural disasters, fire)
• Server failure (failed CPU, bad network card)
• I/O subsystem failure (disk failure, controller failure)
• Software/Data corruption (application bugs, OS binary corruptions)
• User Error (shutdown a SQL service, dropped a table)
4
5. Enterprises need HA
§ HA - High Availability of the Database Service
• Sustain Database Service failure if it goes down
• Sustain Physical Hardware Failures
• Sustain Data/Storage Failures
• 100% Data Guarantee
§ Goal
• Reduce Mean Time To Recover (MTTR) or Recovery Time Objective (RTO)
• Typically driven by SLAs
5
6. Planning a High Availability Strategy
§ Requirements
• Recovery Time Objective (RTO)
• What does 99.99% availability really mean?
• Recovery Point Objective (RPO)
• Zero data lost?
• HA vs. DR requirements
§ Evaluating a technology
• What’s the cost for implementing the technology?
• What’s the complexity of implementing, and managing the technology?
• What’s the downtime potential?
• What’s the data loss exposure?
Availability %
Downtime / Year
Downtime / Month *
Downtime / week
"Two Nines" - 99%
3.65 Days
7.2 Hours
1.69 Hours
"Three Nines" - 99.9%
8.76 Hours
43.2 Minutes
10.1 Minutes
"Four Nines" - 99.99%
52.56 Minutes
4.32 Minutes
1.01 Minutes
"Five Nines" - 99.999%
5.26 Minutes
25.9 Seconds
6.06 Seconds
* Using a 30 day month
6
7. Enterprises need DR
§ DR – Disaster Recovery for your site
• Overcome Complete Site Failure
• Closest if not 100% Data Guarantee expected
• Some data loss may be acceptable
• Key Metrics
• RTO – Recovery Time Objective
• Time to Recover the service
• RPO – Recovery Point Objective
7
8. Enterprises also need Scale UP
§ Scale UP – Throughput increases with more resources given in the
same VM
§ Though in reality limited by Amdahl’s law
8
9. Enterprises also need Scale Out
§ Scale Out – Throughput increases with more resources given via
more nodes (VMs)
§ Typically Shared Nothing architecture (few Shared ‘something’)
§ Often results in “partitions” or “shards”
9
10. Scale Out - For Reads
§ Scale Out or Multi-Node Scaling for Reads
§ Online retailer Use Case
§ 99% Reads and 1% Actual Write transactions
10
11. Scale Out - For Writes
§ Scale Out or Multi-nodes for Writes
§ Example Use case: 24/7 Booking system
§ Constant booking/changes/updates happening
11
12. CAP Theorem
§ Consistency
• all nodes see the same data at the same time
§ Availability
• Guarantee that every request receives a response about whether it was
successful or failed
§ Partition Tolerance
• the system continues to operate despite arbitrary message loss or failure of
part of the system
12
15. VM Mobility
§ Server Maintenance
• VMware vSphere® vMotion® and
VMware vSphere Distributed
Resource Scheduler (DRS)
Maintenance Mode
• Migrate running VMs to other
servers in the pool
• Automatically distribute workloads
for optimal performance
Key Benefits
§ Storage Maintenance • Eliminate downtime for common
maintenance
• VMware vSphere® Storage vMotion
• No application or end user impact
• Migrate VM disks to other storage
targets without disruption • Freedom to perform maintenance
whenever desired
15
16. VMware vSphere High Availability (HA)
§ Protection against host or operating system failure
• Automatic restart of virtual machines on any available host in cluster
• Provides simple and reliable first line of defense for all databases
• Minutes to restart
• OS and application independent, does not require complex configuration
or expensive licenses
16
17. App-Aware HA Through Health Monitoring APIs
§ Leverage third-party solutions that integrate with VMware HA
(for example, Symantec ApplicationHA)
1
Database Health Monitoring
VMware HA • Detect database service failures inside VM
App
Restart 3
2
2 APP APP
Database Service Restart Inside VM
OS OS
• App start / stop / restart inside VM
1 • Automatic restart when app problem detected
3
Integration with VMware HA
• VMware HA automatically initiated when
• App restart fails inside VM
• Heartbeat from VM fails
17
18. Simple, Reliable DR with VMware SRM
§ Provide the simplest and most reliable disaster protection and site
migration for all applications
§ Provide cost-efficient replication of applications to failover site
§ Simplify management of recovery and migration plans
§ Replace manual run books with centralized recovery plans
§ From weeks to minutes to set up new plan
§ Automate failover and migration
processes for reliable recovery
§ Provide for fast, automated failover
§ Enable non-disruptive testing
§ Automate failback processes
18
19. High Availability Options through Virtualization Technologies
PostgreSQL
Streaming
Hardware Failure Tolerance
Replication
Continuous
VMotion
VMware FT
Automated (Planned Downtime)
Restart
RedHat/OS Cluster
VMware HA
Unprotected
0% 10% 100%
Application Coverage
§ Clustering too complex and expensive for most applications
§ VMware HA and FT provide simple, cost-effective availability
§ VMotion provides continuous availability against
planned downtime
19
21. PostgreSQL Replication
§ Single master, multi-slave
§ Cascading slave possible with vFabric Postgres 9.2
§ Mechanism based on WAL (Write-Ahead Logs)
§ Multiple modes and multiple recovery ways
• Warm standby
• Asynchronous hot standby
• Synchronous hot standby
§ Slaves can perform read operations optionally
• Good for read scale
§ Node failover, reconnection possible
21
22. File-based replication
§ File-based recovery method using WAL archives
§ Master node sends WAL files to archive once completed
§ Slave node recovers those files automatically
§ Some delay for the information recovered on slave
• Usable if application can lose some data
• Good performance, everything is scp/rsync/cp-based
• Timing when WAL file is sent can be controlled
vPG
ile slave 1
WAL file WAL f
vPG
master
WAL Archive
disk vPG
slave 2
22
23. Asynchronous replication
§ WAL record-based replication
§ Good balance performance/data loss
• Some delay possible for write-heavy applications
• Data loss possible if slaves not in complete sync due to delay
§ Possible to connect a slave to a master or a slave (cascading
mode)
vPG Slave 1 Slave 1-1
master
WAL shipping
Slave 2 Slave 1-2
23
24. Synchronous mode
§ COMMIT-based replication
• Only one slave in sync with master
• Master waits that transaction COMMIT happens on sync slave, then commits
§ No data loss based on transaction commit
• Performance impact
• Good for critical applications
§ Cascading slaves are async
async
vPG Slave 1 Slave 1-1
master
WAL shipping
Slave 2 Slave 1-2
24
26. Node failover (1)
§ Same procedure for all the replication modes
vPG
Slave
master
§ Failover procedure
Promotion
• Connect to slave VM
ssh postgres@$SLAVE_IP
• Promote the slave
pg_ctl promote
• recovery.conf renamed to recovery.done in $PGDATA
• Former slave able to run write queries
26
27. Node failover (2)
§ Locate archive disk to a new slave node
• Recreate new virtual disk on new node
• Update restore_command in recovery.conf of the remaining slaves
• Update archive_command in postgresql.conf of promoted slave
• Copy WAL files from remaining archive disk to prevent SPOF after loss of
master
27
28. Node reconnection
§ In case a previously failed node is up again
old
Promoted
master
Slave
Reconnect
§ Reconnection procedure
• Connect to old master VM
ssh postgres@$MASTER_IP
• Create recovery.conf depending on recovery mode wanted
recovery_target_timeline = ‘latest’
standby_mode = on
restore_command = ’scp $SLAVE_IP:/archive/%f %p’
primary_conninfo = 'host=$SLAVE_IP application_name=$OLD_NAME’
• Start node
service postgresql start
• Important! Retrieving WAL is necessary for timeline switch
28
29. Additional tips
§ DB and server UI
• Usable as normal, cannot create objects on slave of course
§ wal_level
• ‘archive’ for archive only recovery
• ‘hot_standby’ for read-queries on slaves
§ pg_stat_replication to get status of connected slaves
postgres=# SELECT pg_current_xlog_location(),
application_name,
sync_state,
flush_location
FROM pg_stat_replication;
pg_current_xlog_location | application_name | sync_state | flush_location
--------------------------+------------------+------------+----------------
0/5000000 | slave2 | async | 0/5000000
0/5000000 | slave1 | sync | 0/5000000
(2 rows)
29
31. Single Data Center Deployment
Highly Available PostgreSQL database server with HA from virtualization environment
DNS Name
Applications
Site 1
§ Easy to setup with one click HA
§ Handles CPU/Memory hardware issues
§ Requires Storage RAID 1 for storage protection (atleast)
§ RTO in couple of minutes
31
32. vSphere HA with PostgreSQL 9.2 Streaming Replication)
§ Protection against HW/SW failures and DB corruption
§ Storage flexibility
(FC, iSCSI, NFS)
§ Compatible w/ vMotion,
DRS, HA
§ RTO in few seconds
§ vSphere HA + Streaming Replication
• Master generally restarted with vSphere HA
• When Master is unable to recover, the Replica can be promoted to master
• Reduces synchronization time
after VM recovery
32
33. Single Data Center Deployment
Highly Available PostgreSQL database server with synchronous replication
Virtual IP or
DNS or
pgPool or
Applications
pgBouncer
Site 1
§ Synchronous Replication within Data Center
§ Low Down Time (lower than HA)
§ Automated Failover for hardware issues including Storage
33
34. Multi-site Data Center Deployment
Replication across Data Centers with PostgreSQL for Read Scaling/DR
Applications Virtual IP or
Site 2
pgPool or
pgBouncer
Site 1
§ Synchronous Replication within Data Center
§ Asynchronous replication across data enters
§ Read Scaling (Application Driver ) Site 3
34
35. Multi-site Data Center Deployment
Replication across Data Centers with Write Scaling (requires sharding)
Applications Virtual IP or
Site 2
pgPool or
pgBouncer
Site 1
§ Each Site has its own shard, its synchronous replica and
asynchronous replicas of other sites
§ Asynchronous replication across data enters
Site 3
§ HA/DR built-in
§ Sharding is application driven
35
36. Hybrid Cloud
Hybrid Cloud Scaling for Fluctuating Read peaks
Virtual IP or
pgPool or
pgBouncer
Applications
Site 1
Cascaded
Read
Replicas
§ Many times reads go up to 99% of workload
§ (Example a sensational story that every one wants to read)
§ Synchronous Replication within Data Center
§ Asynchronous Replica slaves within Data Center and on Hybrid Clouds
§ More replicas are spun up when load increases and discarded when it decreases
36