Achieving PCI, HIPPA, and GDPR compliance are interesting challenges that MongoDB DBAs encounter throughout all firms, am I right?
Available MongoDB Security features such as authentication, access control, and encryption, to secure your MongoDB deployments with Opensource choices.
This presentation is designed to provide a comprehensive overview of the top 10 features of MySQL 8.0, explaining why they are advantageous and how they will improve the MySQL experience for users. Furthermore, this presentation will provide a timeline for users to plan for and upgrade from MySQL 5.7, which will reach its end of life by October 2023.
Recording available YouTube Channel: https://www.youtube.com/c/Mydbops?app=desktop
MySQL replication has evolved a lot in 5.6 ,5.7 and 8.0. This presentation focus on the changes made in parallel replication. It covers MySQL 8.0. It was presented at Mydbops database meetup on 04-08-2016 in Bangalore.
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)Jean-François Gagné
To get better replication speed and less lag, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK. But fully benefiting from this feature is not as simple as just enabling it.
In this talk, I explain in detail how this feature works. I also cover how to optimize parallel replication and the improvements made in MySQL 8.0 and back-ported in 5.7 (Write Sets), greatly improving the potential for parallel execution on replicas (but needing RBR).
Come to this talk to get all the details about MySQL 5.7 and 8.0 Parallel Replication.
MySQL and MariaDB though they share the same roots for replication .They support parallel replication , but they diverge the way the parallel replication is implemented.
This presentation is designed to provide a comprehensive overview of the top 10 features of MySQL 8.0, explaining why they are advantageous and how they will improve the MySQL experience for users. Furthermore, this presentation will provide a timeline for users to plan for and upgrade from MySQL 5.7, which will reach its end of life by October 2023.
Recording available YouTube Channel: https://www.youtube.com/c/Mydbops?app=desktop
MySQL replication has evolved a lot in 5.6 ,5.7 and 8.0. This presentation focus on the changes made in parallel replication. It covers MySQL 8.0. It was presented at Mydbops database meetup on 04-08-2016 in Bangalore.
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)Jean-François Gagné
To get better replication speed and less lag, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK. But fully benefiting from this feature is not as simple as just enabling it.
In this talk, I explain in detail how this feature works. I also cover how to optimize parallel replication and the improvements made in MySQL 8.0 and back-ported in 5.7 (Write Sets), greatly improving the potential for parallel execution on replicas (but needing RBR).
Come to this talk to get all the details about MySQL 5.7 and 8.0 Parallel Replication.
MySQL and MariaDB though they share the same roots for replication .They support parallel replication , but they diverge the way the parallel replication is implemented.
MariaDB: in-depth (hands on training in Seoul)Colin Charles
MariaDB: in-depth is training that was conducted for partners selling/deploying MariaDB in Seoul. Its a practical hands-on introduction that can be completed in 1-day.
This presentation shortly describes key features of Apache Cassandra. It was held at the Apache Cassandra Meetup in Vienna in January 2014. You can access the meetup here: http://www.meetup.com/Vienna-Cassandra-Users/
MySQL performance can be improved by tuning queries, server options, and hardware. Traditionally it was an area of responsibility for three different roles: Development, DBA, and System Administrators. Now DevOps handle these all. But there is a gap. Knowledge gained by MySQL DBAs after years or focusing on a single product is hard to gain when you focus on more than one. This is why I am doing this session. I will show a minimal but most effective set of options to improve MySQL performance. For illustrations, I will use real user stories gained from my Support experience and Percona Kubernetes operators for PXC and MySQL.
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdfJesmar Cannao'
ProxySQL is well-affirmed into thousands of production environments for the features we all know: multiplexing, query routing, and rewriting to name a few.
Let's go through those use cases which maybe are the least common: from keeping malicious eyes away from your production data to rebuilding your non-production environment, from stopping having hanging transactions to monitor your instance, from query firewalling to changing ProxySQL configuration without a single line of SQL!
When the sky is the limit, ProxySQL gives you some boost!
Understanding the architecture of MariaDB ColumnStoreMariaDB plc
MariaDB ColumnStore extends MariaDB Server, a relational database for transaction processing, with distributed columnar storage and parallel query processing for scalable, high-performance analytical processing. This session helps MariaDB users understand how MariaDB ColumnStore works and why it’s needed for more demanding analytical workloads, and covers:
Use cases
Query processing
Bulk data insertion
Distributed partitions
Query optimization
Amazon Aurora services are MySQL and PostgreSQL -compatible relational database engines with the speed, reliability, and availability of high-end commercial databases at one-tenth the cost. This session introduces you to Amazon Aurora, explores the capabilities and features of Aurora, explains common use cases, and helps you get started with Aurora.
When does InnoDB lock a row? Multiple rows? Why would it lock a gap? How do transactions affect these scenarios? Locking is one of the more opaque features of MySQL, but it’s very important for both developers and DBA’s to understand if they want their applications to work with high performance and concurrency. This is a creative presentation to illustrate the scenarios for locking in InnoDB and make these scenarios easier to visualize. I'll cover: key locks, table locks, gap locks, shared locks, exclusive locks, intention locks, insert locks, auto-inc locks, and also conditions for deadlocks.
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법Ji-Woong Choi
MySQL 소개
간략한 소개
version history
MySQL 사용처
제품 군 변화
시장 변화
MySQL 구성
MySQL 클라이언트 / 서버 개념
클라이언트 프로그램
MySQL 설치
MySQL 버전
MySQL 설치
MySQL 환경 설정
환경설정, 변수 설정
MySQL 스토리지 엔진 소개
MySQL tuning 소개 및 방법
데이터 백업/복구 방법
백업
복구
MySQL Upgrade
Have you ever needed to get some additional write throughput from MySQL ? If yes, you probably found that setting sync_binlog to 0 (and trx_commit to 2) gives you an extra performance boost. As all such easy optimisation, it comes at a cost. This talk explains how this tuning works, presents its consequences and makes recommendations to avoid them. This will bring us to the details of how MySQL commits transactions and how those are replicated to slaves. Come to this talk to learn how to get the benefit of this tuning the right way and to learn some replication internals.
Jane Uyvova
Senior Solutions Architect, MongoDB
March 21, 2017
MongoDB Evenings San Francisco
Learn how easy it is to set up, operate, and scale your MongoDB deployments in the cloud with MongoDB Atlas.
24시간 365일 서비스를 위한 MySQL DB 이중화.
MySQL 이중화 방안들에 대해 알아보고 운영하면서 겪은 고민들을 이야기해 봅니다.
목차
1. DB 이중화 필요성
2. 이중화 방안
- HW 이중화
- MySQL Replication 이중화
3. 이중화 운영 장애
4. DNS와 VIP
5. MySQL 이중화 솔루션 비교
대상
- MySQL을 서비스하고 있는 인프라 담당자
- MySQL 이중화에 관심 있는 개발자
Percona Live 2021 - MongoDB Security FeaturesJean Da Silva
When we speak about security, the actual reality is that companies need to comply with multiples frameworks and regulations, and assessing which rules apply to each organization is no easy feat.
Over the talk, we will revisit the security feature we can implement in the #MongoDB environment. The aim is to provide further information on what you can use to help your company with future security implementations.
The topics presented will be:
* Authentication
* Authorization
* TLS/SSL
* External Authentication
* Auditing
* Log Redaction
* Encryption – Data at Rest and Client Field Encryption.
Speaker: Jean da Silva – Percona
MariaDB: in-depth (hands on training in Seoul)Colin Charles
MariaDB: in-depth is training that was conducted for partners selling/deploying MariaDB in Seoul. Its a practical hands-on introduction that can be completed in 1-day.
This presentation shortly describes key features of Apache Cassandra. It was held at the Apache Cassandra Meetup in Vienna in January 2014. You can access the meetup here: http://www.meetup.com/Vienna-Cassandra-Users/
MySQL performance can be improved by tuning queries, server options, and hardware. Traditionally it was an area of responsibility for three different roles: Development, DBA, and System Administrators. Now DevOps handle these all. But there is a gap. Knowledge gained by MySQL DBAs after years or focusing on a single product is hard to gain when you focus on more than one. This is why I am doing this session. I will show a minimal but most effective set of options to improve MySQL performance. For illustrations, I will use real user stories gained from my Support experience and Percona Kubernetes operators for PXC and MySQL.
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdfJesmar Cannao'
ProxySQL is well-affirmed into thousands of production environments for the features we all know: multiplexing, query routing, and rewriting to name a few.
Let's go through those use cases which maybe are the least common: from keeping malicious eyes away from your production data to rebuilding your non-production environment, from stopping having hanging transactions to monitor your instance, from query firewalling to changing ProxySQL configuration without a single line of SQL!
When the sky is the limit, ProxySQL gives you some boost!
Understanding the architecture of MariaDB ColumnStoreMariaDB plc
MariaDB ColumnStore extends MariaDB Server, a relational database for transaction processing, with distributed columnar storage and parallel query processing for scalable, high-performance analytical processing. This session helps MariaDB users understand how MariaDB ColumnStore works and why it’s needed for more demanding analytical workloads, and covers:
Use cases
Query processing
Bulk data insertion
Distributed partitions
Query optimization
Amazon Aurora services are MySQL and PostgreSQL -compatible relational database engines with the speed, reliability, and availability of high-end commercial databases at one-tenth the cost. This session introduces you to Amazon Aurora, explores the capabilities and features of Aurora, explains common use cases, and helps you get started with Aurora.
When does InnoDB lock a row? Multiple rows? Why would it lock a gap? How do transactions affect these scenarios? Locking is one of the more opaque features of MySQL, but it’s very important for both developers and DBA’s to understand if they want their applications to work with high performance and concurrency. This is a creative presentation to illustrate the scenarios for locking in InnoDB and make these scenarios easier to visualize. I'll cover: key locks, table locks, gap locks, shared locks, exclusive locks, intention locks, insert locks, auto-inc locks, and also conditions for deadlocks.
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법Ji-Woong Choi
MySQL 소개
간략한 소개
version history
MySQL 사용처
제품 군 변화
시장 변화
MySQL 구성
MySQL 클라이언트 / 서버 개념
클라이언트 프로그램
MySQL 설치
MySQL 버전
MySQL 설치
MySQL 환경 설정
환경설정, 변수 설정
MySQL 스토리지 엔진 소개
MySQL tuning 소개 및 방법
데이터 백업/복구 방법
백업
복구
MySQL Upgrade
Have you ever needed to get some additional write throughput from MySQL ? If yes, you probably found that setting sync_binlog to 0 (and trx_commit to 2) gives you an extra performance boost. As all such easy optimisation, it comes at a cost. This talk explains how this tuning works, presents its consequences and makes recommendations to avoid them. This will bring us to the details of how MySQL commits transactions and how those are replicated to slaves. Come to this talk to learn how to get the benefit of this tuning the right way and to learn some replication internals.
Jane Uyvova
Senior Solutions Architect, MongoDB
March 21, 2017
MongoDB Evenings San Francisco
Learn how easy it is to set up, operate, and scale your MongoDB deployments in the cloud with MongoDB Atlas.
24시간 365일 서비스를 위한 MySQL DB 이중화.
MySQL 이중화 방안들에 대해 알아보고 운영하면서 겪은 고민들을 이야기해 봅니다.
목차
1. DB 이중화 필요성
2. 이중화 방안
- HW 이중화
- MySQL Replication 이중화
3. 이중화 운영 장애
4. DNS와 VIP
5. MySQL 이중화 솔루션 비교
대상
- MySQL을 서비스하고 있는 인프라 담당자
- MySQL 이중화에 관심 있는 개발자
Percona Live 2021 - MongoDB Security FeaturesJean Da Silva
When we speak about security, the actual reality is that companies need to comply with multiples frameworks and regulations, and assessing which rules apply to each organization is no easy feat.
Over the talk, we will revisit the security feature we can implement in the #MongoDB environment. The aim is to provide further information on what you can use to help your company with future security implementations.
The topics presented will be:
* Authentication
* Authorization
* TLS/SSL
* External Authentication
* Auditing
* Log Redaction
* Encryption – Data at Rest and Client Field Encryption.
Speaker: Jean da Silva – Percona
Low Hanging Fruit, Making Your Basic MongoDB Installation More SecureMongoDB
Your MongoDB Community Edition database can probably be a lot more secure than it is today, since Community Edition provides a wide range of capabilities for securing your system, and you are probably not using them all. If you are worried about cyber-threats, take action reduce your anxiety!
MongoDB .local Bengaluru 2019: New Encryption Capabilities in MongoDB 4.2: A ...MongoDB
Many applications with high-sensitivity workloads require enhanced technical options to control and limit access to confidential and regulated data. In some cases, system requirements or compliance obligations dictate a separation of duties for staff operating the database and those who maintain the application layer. In cloud-hosted environments, certain data are sometimes deemed too sensitive to store on third-party infrastructure. This is a common pain for system architects in the healthcare, finance, and consumer tech sectors — the benefits of managed, easily expanded compute and storage have been considered unavailable because of data confidentiality and privacy concerns.
This session will take a deep dive into new security capabilities in MongoDB 4.2 that address these scenarios, by enabling native client-side field-level encryption, using customer-managed keys. We will review how confidential data can be securely stored and easily accessed by applications running on MongoDB. Common query design patterns will be presented, with example code demonstrating strong end-to-end encryption in Atlas or on-premise. Implications for developers and others designing systems in regulated environments will be discussed, followed by a Q&A with senior MongoDB security engineers.
Securing Your Enterprise Web Apps with MongoDB Enterprise MongoDB
Speaker: Jay Runkel, Principal Solution Architect, MongoDB
Level: 200 (Intermediate)
Track: Operations
When architecting a MongoDB application, one of the most difficult questions to answer is how much hardware (number of shards, number of replicas, and server specifications) am I going to need for an application. Similarly, when deploying in the cloud, how do you estimate your monthly AWS, Azure, or GCP costs given a description of a new application? While there isn’t a precise formula for mapping application features (e.g., document structure, schema, query volumes) into servers, there are various strategies you can use to estimate the MongoDB cluster sizing. This presentation will cover the questions you need to ask and describe how to use this information to estimate the required cluster size or cloud deployment cost.
What You Will Learn:
- How to architect a sharded cluster that provides the required computing resources while minimizing hardware or cloud computing costs
- How to use this information to estimate the overall cluster requirements for IOPS, RAM, cores, disk space, etc.
- What you need to know about the application to estimate a cluster size
Securing MongoDB to Serve an AWS-Based, Multi-Tenant, Security-Fanatic SaaS A...MongoDB
MongoDB introduces new capabilities that change the way micro-services interact with the database, capabilities that are either absent or exist only partially in high-end commercial databases such as Oracle. In this session I will share from my experiences building a cloud-based, multi-tenant SaaS application with extreme security requirements. We will cover topics including considerations for storing multi-tenant data in the database, best practices for authentication and authorization, and performance considerations specific to security in MongoDB.
Webinar slides: How to Secure MongoDB with ClusterControlSeveralnines
Watch the slides of our webinar on “How to secure MongoDB with ClusterControl” and find out about the essential steps necessary to secure MongoDB and how to verify if your MongoDB instance is safe.
The recent MongoDB ransom hack caused a lot of damage and outages, while it could have been prevented with maybe two or three simple configuration changes. MongoDB offers a lot of security features out of the box, however it disables them by default.
In this webinar, we explain which configuration changes are necessary to enable MongoDB’s security features, and how to test if your setup is secure after enablement. We also demonstrate how ClusterControl enables security on default installations. And we cover how to leverage the ClusterControl advisors and the MongoDB Audit Log to constantly scan your environment, and harden your security even more.
AGENDA
What is the MongoDB ransom hack?
What other security threats are valid for MongoDB?
How to enable authentication / authorisation
How to secure MongoDB from ransomware
How to scan your system
ClusterControl MongoDB security advisors
Live Demo
SPEAKER
Art van Scheppingen is a Senior Support Engineer at Severalnines. He’s a pragmatic MySQL and Database expert with over 15 years experience in web development. He previously worked at Spil Games as Head of Database Engineering, where he kept a broad vision upon the whole database environment: from MySQL to Couchbase, Vertica to Hadoop and from Sphinx Search to SOLR. He regularly presents his work and projects at various conferences (Percona Live, FOSDEM) and related meetups.
Eagle6 is a product that use system artifacts to create a replica model that represents a near real-time view of system architecture. Eagle6 was built to collect system data (log files, application source code, etc.) and to link system behaviors in such a way that the user is able to quickly identify risks associated with unknown or unwanted behavioral events that may result in unknown impacts to seemingly unrelated down-stream systems. This session is designed to present the capabilities of the Eagle6 modeling product and how we are using MongoDB to support near-real-time analysis of large disparate datasets.
Webinar: Securing your data - Mitigating the risks with MongoDBMongoDB
In this webinar, we walked through examples of the general security threats to databases. And we looked at how you can mitigate them for MongoDB deployments.
Webinar: Compliance and Data Protection in the Big Data Age: MongoDB Security...MongoDB
Data security and privacy are critical concerns in today’s connected world. Data analyzed from new sources such as social media, logs, mobile devices and sensor networks has become as sensitive as traditional transaction data generated by back-office systems. For this reason, big data technologies must evolve to meet the regulatory compliance standards demanded by industry and government. This session provides an overview of MongoDB’s security architecture, including authentication, authorization, auditing and encryption, collectively designed to to defend, detect and control access to valuable online big data.
Tips on Securing Drupal Sites - DrupalCamp Atlanta (DCA)cgmonroe
This is an updated version of this talk given at DrupalCamp Atlanta (DCA)
This presentation is an overview / case study of things learned by experiencing GDPR Security audits, DoS attacks, brute force login attacks, annoying robot crawlers, and hackers doing security probes.
The session will cover the following main topics with tips on how to protected against each of these.
An overview of security threats
Server Level Attacks
Code Level Attacks
User Access Attacks
Internal Attacks
Some suggestions on developing a security plan
People attending should come away with useful knowledge (modules, best practices, sites that help, front end tools and the like) that will help secure their sites.
We're building a storage engine for MongoDB that provides encryption at rest. When we first set out to do this, the questions were many: how do you protect database encryption keys in a distributed environment, where all the code is open source? Can you optimize performance despite the extra steps of encryption and decryption? And most importantly, how do you make the protection mechanisms easy-to-use yet secure? This talk covers the requirements we gathered, the issues we faced, and the design decisions we made. It is aimed at those interested in security, storage engines, and the engineering process.
Securing Your Deployment with MongoDB and Red Hat's Identity Management in Re...MongoDB
MongoDB and Red Hat have collaborated to deliver an integrated solution for securing MongoDB deployments. Red Hat's proven security infrastructure adds extra protection to MongoDB with standards-based identity management featuring centralization of user, password, and certificate information. MongoDB and Red Hat team members present what you need to know to secure your systems, including an overview of Red Hat's Identity Management in Red Hat Enterprise Linux and MongoDB-RHEL security architecture.
One of MongoDB’s primary attractions for developers is that it gives them the ability to start application development without needing to define a formal, up-front schema. Operations teams appreciate the fact that they don't need to perform a time-consuming schema upgrade operation every time the developers need to store a different attribute.
Some projects reach a point where it's necessary to define rules on what's being stored in the database. This webinar explains how MongoDB 3.2 allows that document validation work to be performed by the database rather than in the application code.
This webinar focuses on the benefits of using document validation: how to set up the rules using the familiar MongoDB Query Language and how to safely roll it out into an existing, mature production environment.
Learn from the dozens of large-scale deployments how to get the most out of your Kubernetes environment:
- Container images optimization
- Organizing namespaces
- Readiness and Liveness probes
- Resource requests and limits
- Failing with grace
- Mapping external services
- Upgrading clusters with zero downtime
MongoDB World 2019: New Encryption Capabilities in MongoDB 4.2: A Deep Dive i...MongoDB
Many applications with high-sensitivity workloads require enhanced technical options to control and limit access to confidential and regulated data. In some cases, system requirements or compliance obligations dictate a separation of duties for staff operating the database and those who maintain the application layer. In cloud-hosted environments, certain data are sometimes deemed too sensitive to store on third-party infrastructure. This is a common pain for system architects in the healthcare, finance, and consumer tech sectors — the benefits of managed, easily expanded compute and storage have been considered unavailable because of data confidentiality and privacy concerns.
This session will take a deep dive into new security capabilities in MongoDB 4.2 that address these scenarios, by enabling native client-side field-level encryption, using customer-managed keys. We will review how confidential data can be securely stored and easily accessed by applications running on MongoDB. Common query design patterns will be presented, with example code demonstrating strong end-to-end encryption in Atlas or on-premise. Implications for developers and others designing systems in regulated environments will be discussed, followed by a Q&A with senior MongoDB security engineers.
Do any VM's contain a particular indicator of compromise? E.g. Run a YARA signature over all executables on my virtual machines and tell me which ones match.
Similar to Achieving compliance With MongoDB Security (20)
Efficient MySQL Indexing and what's new in MySQL ExplainMydbops
Efficient MySQL Indexing & What's New in MySQL Explain - Mydbops MyWebinar Edition 32
This session will delve into:
• Strategic indexing techniques: Learn how to optimize your MySQL database by implementing effective indexing strategies, including when to avoid fulltext indexes to prevent wasted resources.
• Demystifying the new MySQL Explain: We'll explore the latest enhancements to the MySQL Explain plan's JSON output format. Discover how to store the output in a variable for further analysis – a valuable addition introduced in MySQL 8.3. You'll also learn about the explain_json_format_version variable, which empowers you to choose between different JSON output versions for greater flexibility.
• Live Chat Engagement: We encourage you to actively participate throughout the webinar! Use the chat functionality to ask questions and share your experiences with indexing and Explain.
This webinar is perfect for:
• Database administrators (DBAs)
• Developers
• Anyone seeking to optimize MySQL performance and streamline database queries
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: info@mydbops.com
Visit: https://www.mydbops.com/
Scale your database traffic with Read & Write split using MySQL RouterMydbops
Scale your database traffic with Read & Write split using MySQL Router
This webinar recording dives into the world of MySQL Router and its capabilities for effectively managing high database traffic loads.
You'll learn:
• The challenges of scaling database traffic
• How MySQL Router facilitates read/write splitting
• The benefits of implementing read/write splitting
• Step-by-step demonstrations for configuring MySQL Router for:
1. Static read/write routing for standalone servers
2. Dynamic read/write split for InnoDB Cluster & Replica Set
• A comparison of popular load balancers (MySQL Router, ProxySQL, Maxscale)
Mydbops is a trusted database management and consultancy provider, helping businesses achieve optimal database performance and scalability.
Connect with Mydbops!
Website: https://www.mydbops.com/
Email: info@mydbops.com
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024Mydbops
Title: PostgreSQL Schema Changes with Minimal Downtime using pg_osc
Speaker: Aakash M, Mydbops
Event: PGConf India, 2024
Description:
This presentation explores pg_osc, a tool that enables efficient schema changes in PostgreSQL tables with minimal downtime and locking. It addresses the challenges of traditional ALTER statements and provides a smoother alternative.
Key points covered:
• Introduction to pg_osc and its benefits.
• Limitations of ALTER statements and how pg_osc overcomes them.
• Step-by-step explanation of the pg_osc process.
• Prominent features and considerations for using pg_osc.
• References and resources for further exploration.
Target Audience:
• Database administrators
• Developers working with PostgreSQL
• Anyone interested in optimizing schema changes
This presentation provides valuable insights for anyone seeking to streamline schema modifications in PostgreSQL while minimizing disruptions.
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...Mydbops
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applications by Bhanu Jamwal, Head of Solution Engineering, PingCAP at the Mydbops Opensource Database Meetup 14.
This presentation discusses the challenges in choosing the right database for modern applications, focusing on MySQL alternatives. It highlights the growth of new applications, the need to improve infrastructure, and the rise of cloud-native architecture.
The presentation explores alternatives to MySQL, such as MySQL forks, database clustering, and distributed SQL. It introduces TiDB as a distributed SQL database for modern applications, highlighting its features and top use cases.
Case studies of companies benefiting from TiDB are included. The presentation also outlines TiDB's product roadmap, detailing upcoming features and enhancements.
Mastering Aurora PostgreSQL Clusters for Disaster RecoveryMydbops
The presentation "Mastering Aurora PostgreSQL Clusters for Disaster Recovery" by Bhuvanesh, Co-Founder & CTO of ShellKode, at the Mydbops OpenSource Database Meetup 14 covers advanced topics in managing Aurora PostgreSQL clusters for disaster recovery purposes.
Bhuvanesh discusses key features of Aurora, such as its decoupled storage and compute layers, auto scaling capabilities, and native replication, highlighting its benefits over traditional RDS instances. He also explores Aurora Global Databases, explaining how they enable replication of data across regions for geo-span applications with low latency.
The presentation includes architecture details, such as physical and log replication, and managed failover options for ensuring high availability. Bhuvanesh shares real-world experiences and best practices for managing Aurora clusters, including handling replication lag and TLS certificate management.
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...Mydbops
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open Source Database Meetup 15
Shivji explores the evolution of transactions, implementation challenges, and insights into distributed database environments. Whether you're a database enthusiast or a tech enthusiast, this presentation offers valuable insights into the world of database management.
Contents:
• Historical perspective of transactions
• Implementing transactions
• Challenges and trade-offs in ACID properties
• Distributed transactions in modern databases like Amazon Aurora, DynamoDB, and Google Spanner
Key Takeaways:
• Understanding the evolution of transactions in databases
• Insights into the challenges of implementing ACID properties
• Exploration of distributed transaction models in leading database systems
AWS RDS in MySQL 2023 Vinoth Kanna @ Mydbops OpenSource Database Meetup 15Mydbops
Discover the latest developments in the AWS RDS MySQL ecosystem with Vinoth Kanna, Founding Partner at Mydbops LLP. Explore the exciting new features and enhancements introduced in RDS MySQL in 2023, including support for Group Replication, snapshot upgrades, dedicated log volumes, and performance insights export to CloudWatch. Gain valuable insights into the introduction of new instance types and version releases throughout the year. Stay ahead of the curve by learning about the end-of-life dates for MySQL RDS 5.7 and extended support pricing considerations. Don't miss out on this informative session to deepen your understanding of AWS RDS MySQL and its evolving capabilities.
Data-at-scale-with-TIDB Mydbops Co-Founder Kabilesh PR at LSPE EventMydbops
Explore the world of TiDB with Kabilesh PR, Co-Founder of Mydbops, as he unveils the potential of this open-source distributed SQL database. Dive into the architecture, scalability solutions, and production readiness of TiDB, and discover how it addresses MySQL scalability bottlenecks through sharding. Gain insights into its stateless SQL interface, transactional storage with TiKV, and analytical capabilities with TiFlash. Learn about TiDB's native sharding features, use cases across various industries, and its readiness for production environments. Delve into its limitations and discover how TiDB can transform your data management landscape.
MySQL Transformation Case Study: 80% Cost Savings & Uninterrupted Availabilit...Mydbops
Discover how Mydbops achieved an impressive 80% cost savings and ensured uninterrupted availability through a transformative MySQL database case study. Join Vinoth Kanna RS, Co-Founder of Mydbops, as he shares insights into optimizing infrastructure, enhancing observability, and navigating critical technology decisions. Learn from real-world challenges, innovative solutions, and valuable takeaways for your own database management endeavors.
Insightful session at Mydbops Opensource Database Meetup 14 in Bangalore as our Chief Technology Officer, Manosh Malai, delves deep into the world of MongoDB optimization. In this engaging presentation, Manosh explores the two primary sharding strategies - Vertical and Horizontal, providing valuable insights and real-world use cases. Gain a comprehensive understanding of the fundamentals of MongoDB sharding, including the pros, cons, and practical applications of both Vertical and Horizontal strategies. Explore real-world case studies and performance benchmarks to optimize your MongoDB deployments.
Mastering MongoDB Atlas: Essentials of Diagnostics and Debugging in the Cloud...Mydbops
Diving deep into the essentials of MongoDB Atlas diagnostics and debugging, helps you ensure optimal performance for your cloud-based databases. Join us as we explore key strategies and best practices for effective database management in the cloud environment. Get ready to elevate your MongoDB Atlas experience and unlock the full potential of your cloud databases.
Data Organisation: Table Partitioning in PostgreSQLMydbops
Mohammad Zaid Patel from Mydbops, embarked on a journey through PostgreSQL table partitioning.
✅ Why Data Organization?
Understand the importance and benefits of organized data in databases.
✅ Advantages of Organizing Your Data:
Better retrieval, improved performance, data integrity, and efficient storage.
✅ Data Organization Techniques:
Index creation, data archival, schemas, functional naming, and relationships.
✅ Table Partitioning in PostgreSQL:
Dive into the design technique of dividing large tables for efficient data management.
✅ Types of Table Partitioning:
Range, List, and Hash methods for unique data organization.
✅ Partitioning Techniques in PostgreSQL:
Manual and using pg_partman extension for streamlined partition creation.
✅ Limitations of Table Partitioning:
Considerations and challenges associated with this technique.
✅ Best Practices for Partitioned Table Maintenance:
Tips on choosing the right partition key, understanding query patterns, and more.
#mydbops #postgresql #mywebinar #webinar #data #database #partitioning #dataorganization #queryperformance #indexing #dataarchival #scalability #dataanalysis #pg_partman #databaseperformance #maintenance #dbms #dba #opensource #highavailability
Navigating MongoDB's Queryable Encryption for Ultimate Security - MydbopsMydbops
Explore MongoDB's Queryable Encryption in this in-depth webinar presentation. Learn about CSFLE, Queryable Encryption, and their mechanisms. Dive into DEKs, Key Vault Collections, Cryptographic Tokens, and more. Discover how MongoDB ensures robust security and flexibility in data encryption.
Explore TiDB's architecture, high availability features, and its ability to handle both transactional and analytical workloads.
Discover the role of the Raft consensus algorithm in ensuring data replication and fault tolerance within the system. Learn about practical use cases in SAAS applications, IoT data management, e-commerce, logistics, gaming, fintech, and more.
Get to know the limitations and advantages of TiDB and how it can revolutionize your data management strategy.
Join us on this knowledge-packed journey!
Mastering Database Migration_ Native replication (8.0) to InnoDB Cluster (8.0...Mydbops
Mastering Database Migration_ Native replication (8.0) to InnoDB Cluster (8.0) with Cloning Best Practices
Explore the slides from our recent webinar on 'Mastering Database Migration: MySQL Replication to InnoDB Cluster Using Cloning.' Dive into the world of database migration, InnoDB Cluster, and the power of cloning. Discover best practices and insights shared by experts in the field. Stay updated with the latest trends in database management
Watch the webinar recording https://youtu.be/sc9TYXKAQWw
Visit our Mydbops blog https://www.mydbops.com/blog/ for further insights.
Enhancing Security of MySQL Connections using SSL certificatesMydbops
Enhancing Security of MySQL Connections using SSL certificates
Mydbops MyWebinar Edition 26
In this informative presentation by Mydbops, explore the world of database security as we delve into the steps to fortify your MySQL connections using SSL certificates. Learn about the working of SSL, the benefits of SSL/TLS encryption, the types of certificates available, and the evolution of SSL/TLS in MySQL. Discover why securing your remote connections and data confidentiality is crucial. Plus, find out how to enable SSL connections in MySQL 8.0. Don't miss this opportunity to bolster your MySQL security knowledge.
Watch the webinar recording https://youtu.be/aMSUtQVdFks
Visit our Mydbops blog https://www.mydbops.com/blog/ for further insights.
Exploring the Fundamentals of YugabyteDB - Mydbops Mydbops
Exploring the Fundamentals of YugabyteDB - Mydbops MyWebinar Edition 25
Join us for an enlightening journey into the world of YugabyteDB, a distributed SQL database revolutionizing data management. In this webinar presentation, we delve into the challenges faced by traditional databases, explore the architecture and unique features of YugabyteDB, and showcase its seamless scalability and fault tolerance.
Watch the full recording: https://youtu.be/QtvK-apLBwQ
Visit Mydbops Blogs: https://www.mydbops.com/blog/
Time series in MongoDB - Mydbops Mywebinar Edition 24. - Explore the fascinating world of time series data management in MongoDB with our insightful webinar presentation. Join us as we dive into the intricacies of leveraging MongoDB for time series use cases, discussing best practices, performance optimization techniques, and real-world examples. Discover how MongoDB can empower your applications to efficiently handle time-based data and unlock valuable insights. Don't miss out on this opportunity to enhance your knowledge and stay ahead in the evolving field of data management. Dive into our speaker deck presentation now!
Watch the webinar recording here: https://youtu.be/rwjHRLGZ7pg
Mydbops Blogs: https://www.mydbops.com/blog/
TiDB in a Nutshell - Power of Open-Source Distributed SQL Database - MydbopsMydbops
TiDB in a Nutshell - Open-Source Distributed SQL Database
Immerse yourself in the world of TiDB Architecture with our captivating presentation. Dive deep into the intricacies of TiDB, the distributed SQL database that has redefined data management. Join us as we unravel the architectural brilliance behind TiDB, exploring its key components, data flow, and design principles. Uncover the secrets to exceptional performance, elastic scalability, and rock-solid data consistency. Prepare to be enlightened by the groundbreaking TiDB Architecture that is revolutionizing the industry.
Watch the full webinar here https://youtu.be/aMSUtQVdFks for webinar recording
Mydbops Blogs: https://www.mydbops.com/blog/
High availability is critical for PostgreSQL database systems, especially for organizations that depend on their databases to support their operations. In this presentation, we will explore the different options available for achieving high availability in PostgreSQL.
Harnessing WebAssembly for Real-time Stateless Streaming PipelinesChristina Lin
Traditionally, dealing with real-time data pipelines has involved significant overhead, even for straightforward tasks like data transformation or masking. However, in this talk, we’ll venture into the dynamic realm of WebAssembly (WASM) and discover how it can revolutionize the creation of stateless streaming pipelines within a Kafka (Redpanda) broker. These pipelines are adept at managing low-latency, high-data-volume scenarios.
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
We have compiled the most important slides from each speaker's presentation. This year’s compilation, available for free, captures the key insights and contributions shared during the DfMAy 2024 conference.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
Using recycled concrete aggregates (RCA) for pavements is crucial to achieving sustainability. Implementing RCA for new pavement can minimize carbon footprint, conserve natural resources, reduce harmful emissions, and lower life cycle costs. Compared to natural aggregate (NA), RCA pavement has fewer comprehensive studies and sustainability assessments.
Understanding Inductive Bias in Machine LearningSUTEJAS
This presentation explores the concept of inductive bias in machine learning. It explains how algorithms come with built-in assumptions and preferences that guide the learning process. You'll learn about the different types of inductive bias and how they can impact the performance and generalizability of machine learning models.
The presentation also covers the positive and negative aspects of inductive bias, along with strategies for mitigating potential drawbacks. We'll explore examples of how bias manifests in algorithms like neural networks and decision trees.
By understanding inductive bias, you can gain valuable insights into how machine learning models work and make informed decisions when building and deploying them.
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdffxintegritypublishin
Advancements in technology unveil a myriad of electrical and electronic breakthroughs geared towards efficiently harnessing limited resources to meet human energy demands. The optimization of hybrid solar PV panels and pumped hydro energy supply systems plays a pivotal role in utilizing natural resources effectively. This initiative not only benefits humanity but also fosters environmental sustainability. The study investigated the design optimization of these hybrid systems, focusing on understanding solar radiation patterns, identifying geographical influences on solar radiation, formulating a mathematical model for system optimization, and determining the optimal configuration of PV panels and pumped hydro storage. Through a comparative analysis approach and eight weeks of data collection, the study addressed key research questions related to solar radiation patterns and optimal system design. The findings highlighted regions with heightened solar radiation levels, showcasing substantial potential for power generation and emphasizing the system's efficiency. Optimizing system design significantly boosted power generation, promoted renewable energy utilization, and enhanced energy storage capacity. The study underscored the benefits of optimizing hybrid solar PV panels and pumped hydro energy supply systems for sustainable energy usage. Optimizing the design of solar PV panels and pumped hydro energy supply systems as examined across diverse climatic conditions in a developing country, not only enhances power generation but also improves the integration of renewable energy sources and boosts energy storage capacities, particularly beneficial for less economically prosperous regions. Additionally, the study provides valuable insights for advancing energy research in economically viable areas. Recommendations included conducting site-specific assessments, utilizing advanced modeling tools, implementing regular maintenance protocols, and enhancing communication among system components.
Hierarchical Digital Twin of a Naval Power SystemKerry Sado
A hierarchical digital twin of a Naval DC power system has been developed and experimentally verified. Similar to other state-of-the-art digital twins, this technology creates a digital replica of the physical system executed in real-time or faster, which can modify hardware controls. However, its advantage stems from distributing computational efforts by utilizing a hierarchical structure composed of lower-level digital twin blocks and a higher-level system digital twin. Each digital twin block is associated with a physical subsystem of the hardware and communicates with a singular system digital twin, which creates a system-level response. By extracting information from each level of the hierarchy, power system controls of the hardware were reconfigured autonomously. This hierarchical digital twin development offers several advantages over other digital twins, particularly in the field of naval power systems. The hierarchical structure allows for greater computational efficiency and scalability while the ability to autonomously reconfigure hardware controls offers increased flexibility and responsiveness. The hierarchical decomposition and models utilized were well aligned with the physical twin, as indicated by the maximum deviations between the developed digital twin hierarchy and the hardware.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
An Approach to Detecting Writing Styles Based on Clustering Techniquesambekarshweta25
An Approach to Detecting Writing Styles Based on Clustering Techniques
Authors:
-Devkinandan Jagtap
-Shweta Ambekar
-Harshit Singh
-Nakul Sharma (Assistant Professor)
Institution:
VIIT Pune, India
Abstract:
This paper proposes a system to differentiate between human-generated and AI-generated texts using stylometric analysis. The system analyzes text files and classifies writing styles by employing various clustering algorithms, such as k-means, k-means++, hierarchical, and DBSCAN. The effectiveness of these algorithms is measured using silhouette scores. The system successfully identifies distinct writing styles within documents, demonstrating its potential for plagiarism detection.
Introduction:
Stylometry, the study of linguistic and structural features in texts, is used for tasks like plagiarism detection, genre separation, and author verification. This paper leverages stylometric analysis to identify different writing styles and improve plagiarism detection methods.
Methodology:
The system includes data collection, preprocessing, feature extraction, dimensional reduction, machine learning models for clustering, and performance comparison using silhouette scores. Feature extraction focuses on lexical features, vocabulary richness, and readability scores. The study uses a small dataset of texts from various authors and employs algorithms like k-means, k-means++, hierarchical clustering, and DBSCAN for clustering.
Results:
Experiments show that the system effectively identifies writing styles, with silhouette scores indicating reasonable to strong clustering when k=2. As the number of clusters increases, the silhouette scores decrease, indicating a drop in accuracy. K-means and k-means++ perform similarly, while hierarchical clustering is less optimized.
Conclusion and Future Work:
The system works well for distinguishing writing styles with two clusters but becomes less accurate as the number of clusters increases. Future research could focus on adding more parameters and optimizing the methodology to improve accuracy with higher cluster values. This system can enhance existing plagiarism detection tools, especially in academic settings.
Water billing management system project report.pdfKamal Acharya
Our project entitled “Water Billing Management System” aims is to generate Water bill with all the charges and penalty. Manual system that is employed is extremely laborious and quite inadequate. It only makes the process more difficult and hard.
The aim of our project is to develop a system that is meant to partially computerize the work performed in the Water Board like generating monthly Water bill, record of consuming unit of water, store record of the customer and previous unpaid record.
We used HTML/PHP as front end and MYSQL as back end for developing our project. HTML is primarily a visual design environment. We can create a android application by designing the form and that make up the user interface. Adding android application code to the form and the objects such as buttons and text boxes on them and adding any required support code in additional modular.
MySQL is free open source database that facilitates the effective management of the databases by connecting them to the software. It is a stable ,reliable and the powerful solution with the advanced features and advantages which are as follows: Data Security.MySQL is free open source database that facilitates the effective management of the databases by connecting them to the software.
1. Achieving Compliances with MongoDB
Security
Presented by
MADHU SAI VAVILALA
Associate DB Engineer
Mydbops
2. Agenda
➔ Importance of Data Security
➔ Security Triad (C.I.A)
➔ A.A.A.
➔ Data Encryption at Rest
➔ TLS & SSL
➔ CSFLE
➔ Queryable Encryption
➔ MongoDB Security Recommendations
3. History Of DataBase
➔ Ancient data storage techniques
◆ Examples: Cave painting, Stone crafting, Pen and Paper,
Punched cards
➔ Necessity of data storage devices
◆ To avoid drawback with traditional data storage
techniques
● Latency to get required data
● Less security
● Need more space to store
➔ First non-electronic storage device
◆ Punch cards in Jaccard machines
➔ The first 'real' electronic storage device in 1947 (Manchester
Mark I Williams-Kilburn tube)
4. Importance of Data Security
➔ Ransomware Attack on 22,000 DB’s in 2020
➔ Security less MongoDB instances 308,000 at Q1 2021
➔ Q1 of 2022, 12 % instances peaked in the list
➔ IOT - Security = Internet Of Threats
➔ “ Privacy is not a right, it is an absolute necessity. ”
➔ Data is a costlier assert
➔ Avoid illegal access
➔ Protect from data corruption
5. Data Security Triad (C.I.A)
➔ Confidentiality:
Protection of data against the
unauthorized user.
➔ Integrity:
Prevention of unauthorized and
improper data modification.
➔ Availability:
An authorized user can able to access
the data without any interruption.
Confidentiality
Integrity Availability
C
A
I
Data
Security
Triad
6. A.A.A
➔ Authentication
◆ Verify Identity of User
◆ Internal Authentication
◆ External Authentication
➔ Authorization
◆ Actions performed by the user
◆ Inbuilt Roles
◆ Customized Roles
➔ Auditing
◆ Logging the activities of a user
Auditing
A
u
t
h
e
n
t
i
c
a
t
i
o
n
A
u
t
h
o
r
i
z
a
t
i
o
n
A3
Security
8. Authentication
➔ Purposes of Authentication
◆ Connect a valid user
◆ Verify the proof of user
◆ Safe from the attackers
➔ Authentication Mechanisms
◆ SCRAM (Default)
◆ x.509 Certificate
◆ LDAP
◆ Kerberos
9. Enable Authentication in MongoDB
➔ Start MongoDB without authentication option
in config file.
➔ Connect to the server using the mongo shell
➔ Create an administrator in the admin database
with a root access.
➔ Exit from the mongo shell.
➔ Restart the MongoDB after adding the
authentication option in MongoDB config file.
10. Methods To Authenticate User in MongoDB
❖ mongo "mongodb://User:Password@<host>:<port>/admin"
❖ mongo -u User -p Password --authenticationDatabase admin
❖ db.auth("User", "Password")
➔ Validate user, password and AuthDB.
➔ Check port number
➔ Verify the method of authentication mechanism.
Validations If Authentication Failed
11. Authentication with x.509 Mechanism
➔ Certificate based authentication Mechanism
➔ Use certificates instead of traditional username and pwd
➔ Provides both Internal and External Authentication
➔ x.509 Certificate need a secure TLS/SSL connection
➔ $external is the authenticationDatabase
➔ The subject of client certificate is added in the $external DB
➔ $external is having only user details
12. Configuration x.509 Mechanism
➔ Create a root CA file in PEM format
➔ Create a TLS/SSL certificate
➔ Edit the config file
➔ Get the subject of a client PEM certificate
◆ openssl x509 -inform PEM -subject
-in <Clent PEM file>
➔ Create a user with the above subject in
$external Database
net:
tls:
mode: requireTLS
CAFile: <path to root CA PEM file>
certificateKeyFile: < TLS/SSL key
PEM file>
URI: mongo --tls --tlsCertificateKeyFile <path to client PEM file>
--tlsCAFile <path to root CA PEM file> --authenticationDatabase
'$external' --authenticationMechanism MONGODB-X509
13. Authorization
➔ Given who you are, what can you do?
➔ Allows user to perform the specified actions.
➔ Advantage
◆ Improved efficiency.
➔ Role Based Access Control
◆ Provide access to the user based on the actions performed.
14. Why Role Based Access control ?
Developer Insert Data
DBA Create Users
Data Scientist Read Data
➔ Various users are perform different
actions
➔ No need to provide root access to
all users
➔ Never trust users
15. Roles in MongoDB
Role: A detailed object consists of Actions with
privileges performed on resources.
Actions: Performable Activities.
Resources: The data on which the User going
to do activities.
> use admin
> db.createRole(
{
role: "dropCollectionUser",
privileges: [
{
actions: [ "dropCollection" ],
resource: { db: "DB",
collection: "" }
}
],
roles: []
}
)
18. Key File Authentication
➔ Internal Authentication Mechanism
➔ Security between members of the
replica set
➔ Shared key for each member
➔ To use keyfile authentication we
need to create database admin
users
Host OS
Key File
Key
Challenge/
Response
Host OS
Key File
MongoDB
Host OS
Key File
MongoDB MongoDB
Key
Challenge/
Response
Key
Challenge/
Response
19. Steps to Configure KeyFile Authentication
➔ Create a Key file
◆ openssl rand -base64 756 > <path-to-keyfile>
◆ chmod 400 <path-to-keyfile>
➔ Copy the keyfile to each replica set member.
➔ Modify the config file in each mongodb replica
member.
◆ > vi /etc/mongod.conf
➔ Restart the mongod process.
◆ > systemctl restart mongod
security:
keyFile: <path-to-keyfile>
replication:
replSetName: <replicaSetName>
20. It’s Nice To Know. Is
There Any Feature
Available in MONGODB
For Monitoring The
Users Activities ?
21. Auditing in MongoDB
➔ Track system activity.
➔ Recording user, system, and
application activities.
➔ Audit information must be reviewed.
Auditing
Records
Authentication
And
Authorization
Replica set
And
Sharded Cluster
CRUD
Operations
➔ Add accountability.
➔ Instigate Suspesious Activity.
➔ Monitor dataBase Activity.
Advantages
25. Versions vs Features
➔ Available from 2.6v
➔ MongoDB Enterprise and Atlas
➔ Filter option available from 3.0v
➔ Starting 5.0 a feature available that runtime audit configuration
and filter management.
➔ Starting 5.0 a UUID is added to the audit message.
➔ 5.3v auditLog.compressionMode → used to compress audit log.
➔ 6.0v onwards audit log encryption is available.
26. Achieving Auditing Feature with Community
➔ Percona server for MongoDB
➔ All features are same with MongoDB Enterprise
server
➔ Only Difference is UUID is not added in percona
MongoDB
➔ Free of cost
27. Even if the Data is
Locked by the
Users with PWD, Is
the data really safe
28. Data Encryption
➔ A mechanism that Protects the Data
➔ An information is converted into cipher Text
➔ Hides the exact Meaning
➔ Only authorized parties can understand
MongoDB involves three types of data:
➔ Data at rest
➔ Data in transit
➔ Data in use
30. Data Encryption at Rest
➔ WiredTiger storage engine have native encryption.
➔ cipher algorithm i.e AES256-GCM
➔ This cipher algorithm support for linux.
➔ Involve symmetric key
➔ Options for sourcing master key
◆ Via 3rd party Key Management Appliance using KMIP
(Key Management Interoperability Protocol )
◆ Keyfile on local system (Not recommendable)
➔ Use unique individual keys for every node in a replica set.
31. Configure Data Encryption At Rest on Community
➔ Create a key file
◆ Base64 and 16 or 32 characters
➔ Edit the config file
➔ Add the encryption options
➔ Restart the mongod
security:
enableEncryption: true
encryptionKeyFile: <Certificate file>
encryptionCipherMode: "AES256-GCM"
{"t":{"$date":"2022-11-21T09:39:37.741+00:00"},"s":"I", "c":"STORAGE", "id":29039,
"ctx":"initandlisten","msg":"Encryption keys DB is initialized successfully"}
Validate
Configuring Steps
Note: It is configured in the percona mongo
grep "Encryption" < log file >
32. Ok!! Now the data is
safe at rest.
But what about
safety while Transit ?
33. Encrypting Data in Transit
Data is transacted through
➔ Transport Layer Security (TLS)
➔ Secure Socket Layer (SSL)
TLS/SSL are uses Certificates
➔ PEM files
➔ Issued by the certificate authority
➔ Self-signed certificate
34. Configure TSL/SSL and Connecting
➔ Create CA file
➔ Create PEM key file
➔ Edit the config file
➔ Add the TLS options
➔ Restart the mongod
net:
tls:
mode: requireSSL
PEMKeyFile: <pem file>
CAFile: <Certificate file>
mongo --tls --tlsPEMKeyFile <pem file> --tlsCAFile <CA file> --host
<host name>
Connection URI
Configuring Steps
35. A sample Demo To Implement Transit
Encryption with Percona MongoDB
36. Ok!! Good !
Now the data is safe
at transit.
But what about
safety while Using ?
37. Client Side Field Level Encryption
➔ Simple Definition
◆ The Client will deal with Encryption and
Decryption of Data using a secure key.
◆ Selectively encrypt individual document
fields.
◆ End-to-end encryption
Doctor’s prescription saves the life of patient, Same
way the Encryptions saves the information from
the data breaches.
38. Why CSFLE Is Important ?
➔ In-flight encryption protects all data traversing the network.
◆ But does not encrypt data in-memory or at-rest.
➔ At-rest encryption protects all stored data.
◆ But does not encrypt data in-memory or in-flight.
➔ With client-side encryption, the most sensitive data never
leaves applications in plain text.
➔ All encrypted fields on the client-side remain encrypted over
◆ stored in-memory,
◆ in system logs, at-rest,
◆ in backups – are rendered as ciphertext
39. Keys Involved
➔ Customer Master Key
◆ A root key
◆ Used to Encrypt or Decrypt the DEK
◆ Must be secured with KMS
➔ The Data Encryption Key
◆ A key generated by libmongocrypt
◆ Encrypted using the CMK
◆ Used to encrypt and decrypt data.
This Strategy is based on Envelope Encryption.
40. Process Of FLE
User
KMS
MongoDB Driver
MongoDB
MongoDB Server
1
2 3
4
5
6
db.coll.find({
Pin: “123”
}) Encrypted Search Key:“***”
{ Name: ‘A’,
age: 16
Pin: “***” }
{
Name: ‘A’,
age: 16
Pin: “123”
}
41. Process Of FLE
➔ When the application submits the query, the MongoDB driver first
analyzes it to determine if any encrypted fields are involved in the filter.
➔ Recognizing that the query is against an encrypted field, the driver
requests the field's encryption key from the external key manager.
➔ The key manager returns the keys to the MongoDB driver, which then
encrypts the queired field.
➔ The driver submits the query to the MongoDB server with the encrypted
fields rendered as ciphertext.
➔ The MongoDB server returns the encrypted results of the query to the
driver.
➔ The query results are decrypted with the keys held by the driver, and
returned to the authenticated client as readable plaintext.
42. Availability of CSFLE
➔ Versions: 4.2v +
➔ Divers: 20 + platforms of multiple languages (Node. Js, C, C#,
Python, Java etc.)
➔ Automatic Encryption: MongoDB Atlas and Enterprise versions.
➔ Manual Encryption: MongoDB Atlas and Enterprise versions
and community versions
43. Configure CSFLE on MongoDB Community
➔ Create a key file i.e CMK
➔ Assine the CMK value to LOCAL_KEY variable
➔ Frame ClientSideFieldLevelEncryptionOptions
➔ Connect to the mongo with above options
➔ Creation of Data Encryption Key
➔ Insert a document and encrypt the any field with
encrypt() function.
Configuring Steps
44. A sample Demo To Implement CSFLE
with MongoDB Community Server
45. The data field is encrypted,
well and Good. Then How
to query on the particular
Field even it is having less
cardinality ?
46. Queryable Encryption
➔ Allows user to search their data while it remains encrypted
➔ Data remains encrypted at insert, storage, and query.
➔ A Public Preview Feature
➔ Not recommended for production
Magical Features of Queryable Encryption
➔ Fully randomized encryption
➔ Expressive queries on encrypted data
➔ Client-side encryption
➔ Customer-managed encryption keys
➔ Industry standard cryptography primitives
➔ Field-level encryption
47. Scope of Necessity For QE
With the proliferation of different types of data being transmitted and stored in
the cloud, protecting data is increasingly important for companies.
➔ Bank Applications
➔ Stock brokerage firm
➔ Hospitals
➔ Human Resources
➔ Government Applications
➔ E-commerce applications etc.
48. Encrypted Collection Management
Queryable Encryption uses four data structures:
➔ Three metadata collections
When you create an encrypted collection using Queryable Encryption,
MongoDB creates three metadata collections:
● enxcol_.<collectionName>.esc, referred to as ESC
● enxcol_.<collectionName>.ecc, referred to as ECC
● enxcol_.<collectionName>.ecoc, referred to as ECOC
➔ A field in every document in the encrypted collection called __safeContent__
Note: It is critical that these data structures are not modified or deleted.
49. Write Operations Vs Metadata Collections
When writing a document, each indexed field requires writes to metadata
collections along with one write on the encrypted collection.
Operation ESC ECC ECOC
Insert 1 0 1
Update 1 1 2
Delete 0 1 1
50. Storage Issue with QE
➔ 2x or 3x times extra storage
➔ Depends on the number of indexed fields per document
➔ For every write Associated metadata collections will
update
IndexCompaction
➔ A process that prunes the metadata collections
➔ Should run when the size of ECOC exceeds 1 GB.
➔ To run index compaction,
db.collection.compactStructuredEncryptionData()
51. Availability of Queryable Encryption
➔ Versions: 6.0v
➔ Divers: 20 + platforms of multiple languages (Node.Js, C, C#,
Python, Java etc.)
➔ Automatic Encryption: MongoDB Atlas and Enterprise versions.
➔ Manual Encryption: MongoDB Atlas and Enterprise versions
and community versions.
52. Limitations of QE:
➔ Fully randomized encryption
➔ Manually create a unique Data Encryption Key for each field
➔ Manually run index compaction
➔ Standalone deployments are not supported
➔ Secondary reads are not supported
➔ Encrypted field names are immutable.
➔ Ignores collations
➔ Unique indexes and TTl indexes are not supported.
➔ Don’t support Views, Time Series Collections, or capped collection.
➔ Can’t encrypt _Id
53. Encryption Performance in MongoDB
➔ Data size α Latency
➔ Consider High load condition
◆ Writes 10% to 20% latency
◆ Reads < 5% (negligible)
➔ Encrypting and decrypting require more resources
➔ WiredTiger storage engine is more suitable. Because
◆ Page level encryption
◆ Throughput operations performs on the exact page only
◆ High performance, scalability, and security
54. MongoDB Security Recommendations
➔ Don’t directly expose the DB servers to the internet
➔ Avoid using the default port
➔ Use strong passwords
➔ Authenticate with db.auth()
➔ Never use the security.transitionToAuth option in config file.
➔ Limit the user actions, Use customized roles.
➔ Maintain data encryption.