This document provides an overview of Distributed Replicated Block Device (DRBD), which is a software-based storage solution that mirrors the contents of block devices across multiple nodes in real-time. It describes DRBD's architecture including kernel modules, user space tools, resource configuration, roles, modes, features around efficient synchronization and data verification, and limitations.
In this presentation, I have talked about Resiliency in Azure.
I have also talked about how you can do Azure VM Improvements and Maintenance. Along with that, I have also talked about Disaster Recovery with ASR.
From the outset, Oracle has delivered the industry's most advanced technology to safeguard data where it lives—in the database. Oracle provides a comprehensive portfolio of security solutions to ensure data privacy, protect against insider threats, and enable regulatory compliance for both Oracle and non-Oracle Databases. With Oracle's powerful database activity monitoring and blocking, privileged user and multi-factor access control, data classification, transparent data encryption, consolidated auditing and reporting, secure configuration management, and data masking, customers can deploy reliable data security solutions that do not require any changes to existing applications, saving time and money.
Delta Lake, an open-source innovations which brings new capabilities for transactions, version control and indexing your data lakes. We uncover how Delta Lake benefits and why it matters to you. Through this session, we showcase some of its benefits and how they can improve your modern data engineering pipelines. Delta lake provides snapshot isolation which helps concurrent read/write operations and enables efficient insert, update, deletes, and rollback capabilities. It allows background file optimization through compaction and z-order partitioning achieving better performance improvements. In this presentation, we will learn the Delta Lake benefits and how it solves common data lake challenges, and most importantly new Delta Time Travel capability.
In this presentation, I have talked about Resiliency in Azure.
I have also talked about how you can do Azure VM Improvements and Maintenance. Along with that, I have also talked about Disaster Recovery with ASR.
From the outset, Oracle has delivered the industry's most advanced technology to safeguard data where it lives—in the database. Oracle provides a comprehensive portfolio of security solutions to ensure data privacy, protect against insider threats, and enable regulatory compliance for both Oracle and non-Oracle Databases. With Oracle's powerful database activity monitoring and blocking, privileged user and multi-factor access control, data classification, transparent data encryption, consolidated auditing and reporting, secure configuration management, and data masking, customers can deploy reliable data security solutions that do not require any changes to existing applications, saving time and money.
Delta Lake, an open-source innovations which brings new capabilities for transactions, version control and indexing your data lakes. We uncover how Delta Lake benefits and why it matters to you. Through this session, we showcase some of its benefits and how they can improve your modern data engineering pipelines. Delta lake provides snapshot isolation which helps concurrent read/write operations and enables efficient insert, update, deletes, and rollback capabilities. It allows background file optimization through compaction and z-order partitioning achieving better performance improvements. In this presentation, we will learn the Delta Lake benefits and how it solves common data lake challenges, and most importantly new Delta Time Travel capability.
This presentation walks through the Security and Compliance functionality to customers leveraging Azure as a compute environment. It includes deep-dive references to detailed information on each topic presented.
Organizations are grappling to manually classify and create an inventory for distributed and heterogeneous data assets to deliver value. However, the new Azure service for enterprises – Azure Synapse Analytics is poised to help organizations and fill the gap between data warehouses and data lakes.
This is part 1 of the Azure storage series, where we will build our understanding of Azure Storage, and will also learn about the storage data services, and the types of Azure Storage. Last but not least, we will also touch base on securing storage accounts
In the second part, we will continue with our demo on creating and utilizing the Azure Storage.
This presentation is to understand StreamSets ETL tool.
StreamSets is modern ETL tool designed to process streaming data.
StreamSets has 2 engines, 1 is Data Controller and Data Transformer(Based on Apache Spark).
Migrate a successful transactional database to azureIke Ellis
This slide deck will show you techniques and technologies necessary to take a large, transaction SQL Server database and migrate it to Azure, Azure SQL Database, and Azure SQL Database Managed Instance
This slide deck was provided by Microsoft for a crash course on Microsoft Azure at the Saint Louis Cloud Camp.
If you need a quick high level introduction to Azure and/or the cloud in general, this presentation would serve as a good template.
Part 3 - Modern Data Warehouse with Azure SynapseNilesh Gule
Slide deck of the third part of building Modern Data Warehouse using Azure. This session covered Azure Synapse, formerly SQL Data Warehouse. We look at the Azure Synapse Architecture, external files, integration with Azuer Data Factory.
The recording of the session is available on YouTube
https://www.youtube.com/watch?v=LZlu6_rFzm8&WT.mc_id=DP-MVP-5003170
This presentation is on the DRBD product. At eNovance, we're using it for several years. In those slides, you will find informations on how we use it, use cases and Ninja tricks.
This document has been realized with a lot of feedbacks and thanks to strong knowledges on that technology that eNovance is able to provide.
Accomplishing redundancy on Lustre based PFS with DRBDTyrone Systems
A premier research institute wanted a HPC along with Parallel File System (PFS) to be deployed which would have complete redundancy. The PFS was based on Lustre file system, which didn’t have any redundancy features; this turned a simple HPC deployment to a challenging one for Netweb Technologies.
This presentation walks through the Security and Compliance functionality to customers leveraging Azure as a compute environment. It includes deep-dive references to detailed information on each topic presented.
Organizations are grappling to manually classify and create an inventory for distributed and heterogeneous data assets to deliver value. However, the new Azure service for enterprises – Azure Synapse Analytics is poised to help organizations and fill the gap between data warehouses and data lakes.
This is part 1 of the Azure storage series, where we will build our understanding of Azure Storage, and will also learn about the storage data services, and the types of Azure Storage. Last but not least, we will also touch base on securing storage accounts
In the second part, we will continue with our demo on creating and utilizing the Azure Storage.
This presentation is to understand StreamSets ETL tool.
StreamSets is modern ETL tool designed to process streaming data.
StreamSets has 2 engines, 1 is Data Controller and Data Transformer(Based on Apache Spark).
Migrate a successful transactional database to azureIke Ellis
This slide deck will show you techniques and technologies necessary to take a large, transaction SQL Server database and migrate it to Azure, Azure SQL Database, and Azure SQL Database Managed Instance
This slide deck was provided by Microsoft for a crash course on Microsoft Azure at the Saint Louis Cloud Camp.
If you need a quick high level introduction to Azure and/or the cloud in general, this presentation would serve as a good template.
Part 3 - Modern Data Warehouse with Azure SynapseNilesh Gule
Slide deck of the third part of building Modern Data Warehouse using Azure. This session covered Azure Synapse, formerly SQL Data Warehouse. We look at the Azure Synapse Architecture, external files, integration with Azuer Data Factory.
The recording of the session is available on YouTube
https://www.youtube.com/watch?v=LZlu6_rFzm8&WT.mc_id=DP-MVP-5003170
This presentation is on the DRBD product. At eNovance, we're using it for several years. In those slides, you will find informations on how we use it, use cases and Ninja tricks.
This document has been realized with a lot of feedbacks and thanks to strong knowledges on that technology that eNovance is able to provide.
Accomplishing redundancy on Lustre based PFS with DRBDTyrone Systems
A premier research institute wanted a HPC along with Parallel File System (PFS) to be deployed which would have complete redundancy. The PFS was based on Lustre file system, which didn’t have any redundancy features; this turned a simple HPC deployment to a challenging one for Netweb Technologies.
DRBD + OpenStack (Openstack Live Prague 2016)Jaroslav Jacjuk
Michal Němec, Cooperation Director PROZETA
PROZETA společně s rakouským LINBITem přináší na trh nejrychlejší open-source Software Defined Storage (SDS) řešení postavené na DRBD s nemalým cílem - vytlačit ze své pozice v současnosti nejvyužívanější a často nevyhovující SDS pro OpenStack Ceph. Co stálo za rozhodnutím pustit se do tohoto ambiciózního projektu, který využívá technologii nasazovanou a rozvíjenou po více než 10 let, instalovanou ve více než 250 000 produkčních instancí po celém světě? Jaká úskalí bylo třeba překonat, jaké nástroje jsou přesně k dispozici pro produkční provoz? Jak si na tom nové SDS stojí výkonnostně a jaká je roadmapa do budoucna?
Introduction to Ceph, an open-source, massively scalable distributed file system.
This document explains the architecture of Ceph and integration with OpenStack.
Storage tiering and erasure coding in Ceph (SCaLE13x)Sage Weil
Ceph is designed around the assumption that all components of the system (disks, hosts, networks) can fail, and has traditionally leveraged replication to provide data durability and reliability. The CRUSH placement algorithm is used to allow failure domains to be defined across hosts, racks, rows, or datacenters, depending on the deployment scale and requirements.
Recent releases have added support for erasure coding, which can provide much higher data durability and lower storage overheads. However, in practice erasure codes have different performance characteristics than traditional replication and, under some workloads, come at some expense. At the same time, we have introduced a storage tiering infrastructure and cache pools that allow alternate hardware backends (like high-end flash) to be leveraged for active data sets while cold data are transparently migrated to slower backends. The combination of these two features enables a surprisingly broad range of new applications and deployment configurations.
This talk will cover a few Ceph fundamentals, discuss the new tiering and erasure coding features, and then discuss a variety of ways that the new capabilities can be leveraged.
The primary requirements for OpenStack based clouds (public, private or hybrid) is that they must be massively scalable and highly available. There are a number of interrelated concepts which make the understanding and implementation of HA complex. The potential for not implementing HA correctly would be disastrous.
This session was presented at the OpenStack Meetup in Boston Feb 2014. We discussed interrelated concepts as a basis for implementing HA and examples of HA for MySQL, Rabbit MQ and the OpenStack APIs primarily using Keepalived, VRRP and HAProxy which will reinforce the concepts and show how to connect the dots.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
DRBD is a open source technology for replicating block based data. It can be used to build high availability clusters, software defined storeage for IaaS clouds and long distance replication (disaster recovery).
Introduction to Cassandra and CQL for Java developersJulien Anguenot
This talk will provide a high-level overview of Cassandra, the Cassandra Query Language (CQL) and more specifically the DataStax CQL Java driver. This talk will aim to introduce Java developers tools, techniques and best practices for building Java application leveraging the Cassandra database using CQL3.
Development of concurrent services using In-Memory Data Gridsjlorenzocima
As part of OTN Tour 2014 believes this presentation which is intented for covers the basic explanation of a solution of IMDG, explains how it works and how it can be used within an architecture and shows some use cases. Enjoy
Over the past 1.5 decade our industry has tried to adopt an increased amount of infrastructure automation. We called it Configuration Management, Infrastructure as Code, infrastructure as Software, Provisioning, Orchestration. We learned about Desired State, Idempotence, etc.. We have seen a number of tools become popular; we have seen a number of tools disappear. But over the years we have seen a number of patterns appear and reappear. Patterns that lead to actually getting great benefits out of automation, or just wasting time while missing out on goals. This talk will explain you a number of these patterns which we have frequently encountered in the wild, with their benefits and caveats. We will try to keep this tool agnostic. Your vision might be Clouded, and you might have to take this with a grain of Salt while you play the Chef from the Muppet show the story, all names, characters, and incidents portrayed in this production are fictitious. No identification with actual persons (living or deceased), places, buildings, and products are intended or should be inferred.
Solving Your Backup Needs Using MongoDB Ops Manager, Cloud Manager and AtlasMongoDB
Backup is an important part of your MongoDB deployment. Come and learn about the different offerings MongoDB has to help meet your backup requirements.
MongoDB.local DC 2018: Solving Your Backup Needs Using MongoDB Ops Manager, C...MongoDB
Backup is an important part of your MongoDB deployment. Come and learn about the different offerings MongoDB has to help meet your backup requirements.
MongoDB.local Austin 2018: Solving Your Backup Needs Using MongoDB Ops Manage...MongoDB
Backup is an important part of your MongoDB deployment. Come and learn about the different offerings MongoDB has to help meet your backup requirements.
Do you need to scale your application, share data across cluster, perform massive parallel processing on many JVMs or maybe consider alternative to your favorite NoSQL technology? Hazelcast to the rescue! With Hazelcast distributed development is much easier. This presentation will be useful to those who would like to get acquainted with Hazelcast top features and see some of them in action, e.g. how to cluster application, cache data in it, partition in-memory data, distribute workload onto many servers, take advantage of parallel processing, etc.
Presented on JavaDay Kyiv 2014 conference.
my talk from highload++ 2013 -- talking about scaling compiled applications but from the point of view of scaling up from supporting 1 platform to supporting MANY platforms.
in other words: given an application that supports ubuntu 10.04, what sort of systems, tips, and tricks are needed to help scale support to other ubuntus, redhats, centos, windows, etc.
Replication, Durability, and Disaster RecoverySteven Francia
This session introduces the basic components of high availability before going into a deep dive on MongoDB replication. We'll explore some of the advanced capabilities with MongoDB replication and best practices to ensure data durability and redundancy. We'll also look at various deployment scenarios and disaster recovery configurations.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
3. What is it?
Distributed Replicated Block Device
Software-based, shared-nothing replicated
storage solution mirroring the contents of block
devices
4. What is it?
Distributed Replicated Block Device
Software-based, shared-nothing replicated
storage solution mirroring the contents of block
devices
• In real time
5. What is it?
Distributed Replicated Block Device
Software-based, shared-nothing replicated
storage solution mirroring the contents of block
devices
• In real time
• Transparently
6. What is it?
Distributed Replicated Block Device
Software-based, shared-nothing replicated
storage solution mirroring the contents of block
devices
• In real time
• Transparently
• Synchronously/asynchronously
8. User space admin tools
• drbsetup
• Used to configure the kernel module
• All parameters in command-line
9. User space admin tools
• drbsetup
• Used to configure the kernel module
• All parameters in command-line
• drbdmeta
• Create/dump/restore/modify DRBD metadata
(more on this later)
10. User space admin tools
• drbsetup
• Used to configure the kernel module
• All parameters in command-line
• drbdmeta
• Create/dump/restore/modify DRBD metadata
(more on this later)
• drbdadm
• High-level, frontend for drbdsetup/drbdmeta
• Reads from /etc/drbd.conf
• Has a dry-run option (-d)
12. Resources
• A particular replicated storage device
• Resource name
• DRBD device: virtual block device (major=147).
The associated block device is always
/dev/drbdm (m=minor)
• Disk configuration: local copy of the data
• Network configuration: comms with peer
13. Configuration
Per resource (/etc/drbd.d/mysql.res):
resource mysql {
device minor 0; # /dev/drbd0
disk /dev/sdb;
meta-disk internal;
on alice {
address 192.168.133.111:7000;
}
on bob {
address 192.168.133.112:7000;
}
syncer {
rate 10M; # static resync rate of 10MByte/s
}
}
15. Resource roles
• Primary: read and write ops
• Secondary: receives updates from primary,
disallows any other access.
• Promotion: from secondary to primary
drbdadm primary all
• Demotion: from primary to secondary
drbdadm secondary all
20. Features: efficient synchronization
• Synchronization != replication
• Inconsistent remote dataset during sync
• Useless
• Service in active node unaffected
21. Features: efficient synchronization
• Synchronization != replication
• Inconsistent remote dataset during sync
• Useless
• Service in active node unaffected
• Synchronization and replication happen at
the same time
24. Features: efficient synchronization
• Only one write op per several successive
writes in active node in a block
• Linear access to blocks
• Configure rate of sync
25. Features: efficient synchronization
• Only one write op per several successive
writes in active node in a block
• Linear access to blocks
• Configure rate of sync
• Checksum-based synchronization
29. Features: disk
• Support for disk flushes
• Disk error handling strategies
• Passing
• Masking
• DIY
30. Features: disk
• Support for disk flushes
• Disk error handling strategies
• Passing
• Masking
• DIY
• Deal with outdated data
• DRBD won't promote an outdated
resource -> fencing
33. Split-brain
Split brain is a situation where, due to temporary failure
of all network links between cluster nodes, and possibly
due to intervention by a cluster management software
or human error, both nodes switched to the primary
role while disconnected.
35. Split-brain
• Configurable notifications
• Automatic recovery methods
• Discard modifications on 'younger' primary.
• Discard modifications on 'older' primary.
• Discard modifications on primary with
fewer changes.
• Graceful recovery if one primary had no
changes.
36. Metadata
• Various pieces of information about the data
DRBD keeps in a dedicated area
• The size of the DRBD device
• The generation identifier
• The activity log
• The quick-sync bitmap
40. What it’s not/What it can’t do
• It’s not a backup system
• It can’t add features to upper layers
41. What it’s not/What it can’t do
• It’s not a backup system
• It can’t add features to upper layers
• DRBD cannot auto-detect file system
corruption
• DRBD cannot add active-active clustering
capability to file systems like ext3 or XFS.
43. Limitations
• Only two nodes
• Stacked resources
• Version 9
• There is no automatic failover.
44. Limitations
• Only two nodes
• Stacked resources
• Version 9
• There is no automatic failover.
• Promotion/demotion is manual.
45. Limitations
• Only two nodes
• Stacked resources
• Version 9
• There is no automatic failover.
• Promotion/demotion is manual.
• Needs a CRM to be useful
PACEMAKER FTW
