Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud

•

0 likes•17 views

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud - Sagy Volkov, Lightbits PostgreSQL as a SQL engine can accommodate a very high-transaction rate, but as your data grows and the number of connections and queries increases, there is a challenge for the storage to keep up with the SQL engine. To the rescue comes NVMe over TCP (or NVMe/TCP). Developed by Lightbits Labs in 2016 and donated to the Linux community, it is the next evaluation of using NVMe based storage over TCP Fabric. NVMe/TCP simplifies how you interact with remote NVMe devices (targets) and allows your PostgreSQL storage to consume fast storage very easily. In this session I will explain the core concept of the NVMe/TCP protocol, current storage providers that can use it, how you can consume it in Kubernetes (super easy), and discuss the possibilities of using NVMe/TCP in the cloud. The session will also include a performance comparison of a few storage that are available in AWS and even a live demo of how PostgreSQL can run super fast - warp speed fast - in AWS.

Technology

Sagi Volkov
Distinguished Architect
Lightbits Labs
Run PostgreSQL in Warp Speed
Using NVMe/TCP in the Cloud
November 6, 2023 | Chicago, Illinois

Agenda
● Intro to NVMe/TCP
● Why NVMe/TCP?
● Some performance comparison (and how I took them).
● Questions

NVMe/TCP - history lesson…
▪ NVM Express™ (NVMe) standard released in March 2011
̶ Architecture, command set, and queueing interface for PCIe SSDs
• Optimized for direct attached NVM PCIe® SSDs
• The goal was a single interface that is scalable from client to enterprise
▪ NVMe™ over Fabrics (NVMe-oF™) standard released in June 2016
̶ Extends the architecture, command set, and queueing interface of NVMe to fabric attached
storage
̶ Two initial transport binding specifications
• RDMA (Infiniband, RoCE, iWARP)
• Fibre Channel
▪ TCP transport binding specification (NVMe™/TCP) specified for NVMe-oF in November
of 2018

NVMe/TCP - more history…
▪ Also in 2018, Lightbits Labs introduced the first storage solution that was
designed and built for NVMe/TCP
▪ Added to the Linux kernel in 4.10
▪ Part of all major Linux enterprise distros.
▪ Today: Lightbits, Ceph, Dell, NetApp, Pure Storage and more.

Why NVMe/TCP (or why do we need
another NVMe-oF transport?)
▪ PCIe®
̶ Great for direct attached NVMe SSDs
̶ Does not scale well to large topologies
▪ RDMA (Infiniband, RoCE, iWARP) and Fibre Channel
̶ Provides a high degree of scalability, but requires special networks and hardware
▪ TCP
̶ Ubiquitous (does not require special networks or hardware)
̶ Scalable allowing large scale deployments and operation over long distances
̶ Can provide performance (throughput and latency) that is comparable to direct
attached NVMe SSDs

Things to consider:
● Remember that you can use NVMe/TCP, wherever you
have TCP available. Simple as that.
● Instance in the cloud usually have higher bandwidth for
network than for native storage from the cloud provider.
● Network resources are billed differently depending on
your cloud provider (ingress/egress)

Performance comparison (spec)
▪ Why PostgreSQL?
▪ Why Kubernetes?
▪ Why sysbench? https://github.com/akopytov/sysbench
▪ Why Sherlock? https://github.com/sagyvolkov/sherlock

Performance comparison (spec)
Availability Zone
VPC
18.75
Gbps
R6in.8xlarge
AWS Cloud
I4i.x8large
VPC Subnet
I4i.x8large
I4i.x8large
18.75
Gbps
18.75
Gbps
VPC Subnet
Pod:
Sherlock
(sysbench)
cpu=0
memory=2Gi
PVC
Pod:
Sherlock
(sysbench)
cpu=0
memory=2Gi
PVC
Pod:
Sherlock Stats
cpu=0.1
memory=64Mi

Performance comparison (spec)
Sysbench test
type Threads Lightbits io2 Block Express
75r25w 40 4,116,162 1,255,658
read-only 40 5,655,487 1,798,263
write-only 40 13,778,113 5,131,169
75r25w 75 5,256,168 1,895,664
read-only 75 7,193,175 2,835,951
write-only 75 14,983,822 6,236,878
75r25w 90 5,350,728 1,990,224
read-only 90 7,309,818 2,952,594
write-only 90 15,099,258 6,952,314
● 1 worker - R6in.8x
● 3 PostgreSQL pods in the
worker
● Each pod with
Request/Limit of cpu=12
and memory=2Gi.
● Each pod with 300GB PVC.
● Each databases, 400 tables
of 1M records.

Thanks
https://www.lightbitslabs.com
My twitter: @clusterguru
Sherlock: https://github.com/sagyvolkov/sherlock

Link: https://youtu.be/cegd3Exg05w https://go.dok.community/slack https://dok.community/ Gabriele Bartolini - Vice President/CTO of Cloud Native and Kubernetes, EDB ABSTRACT OF THE TALK Imagine this: you have a virtual infrastructure based on Kubernetes, made up of virtual data centers, possibly spread across multiple Kubernetes clusters and regions. Your infrastructure could even be hosted on premises or on different cloud service providers. Infrastructure as Code is a requirement. You’ve been tasked to run Postgres databases, alongside your applications. The good news is that you can leverage a fully open source stack with Kubernetes, PostgreSQL and the CloudNativePG operator, and deploy your Postgres database in the same way you deploy applications. Join me in this webinar to discover the key role that you have to make this succeed, starting from day 0 through day 2 operations. I’ll share some examples and best practices for running Postgres databases in Kubernetes, before peeking at the new features we are developing for the months to come.

Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...

DoKC

Link: https://youtu.be/Y-1uFVKDfgY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK This talk concerns performing analytical tasks with Apache Superset with ClickHouse as the data backend. ClickHouse is a super fast database for analytical tasks, and Apache Superset is an Apache Software foundation project meant for data visualization and exploration. Performing analytical tasks using this combo is super fast since both the software are designed to be scalable and capable of handling data of petabyte scale.

Analytics with Apache Superset and ClickHouse - DoK Talks #151

DoKC

Link: https://youtu.be/EFaRyl4HmmE https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK If you are running or planning a multi-cloud or even a multi-cluster environment, there are several considerations in implementing a data protection solution – especially if you plan on an organic home-grown, do-it-yourself option. This talk will highlight challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. We will also highlight the deviations in CSI driver implementations of various storage vendors and cloud providers. Finally, we will cover the various recovery options available in the market today. Kubernetes cloud services are popular since they mitigate, but do not eliminate, the difficulties of operating a Kubernetes environment. This is especially true for protecting the stateful configuration and data of your Kubernetes applications, where the inherent high-availability and infrastructure as code are not a substitute for have cloud-native backup and disaster recovery capabilities. Further, many companies now have multi-cloud strategies for their cloud-native applications. These challenges can be addressed with backup applications that are both Kubernetes managed service and multi-cloud aware in order to snapshot, copy, restore, and migrate Kubernetes workloads (resources and data) running on AKS, EKS and GKE. Capturing information from cloud accounts and how the cluster and storage resources are configured allows 1) centralized visibility into all cloud accounts and the clusters and resources in the accounts including for compliance; 2) cross-account, cross-cluster, and cross-region data restores; 3) automation of the cluster and data restores including for Dev, Test, and Production recovery use cases. BIO Sebastian Glab is a Cloud Architect for CloudCasa and he resides in Poland. He is responsible for integrating the different cloud providers with the CloudCasa service, and making sure that all clusters in the cloud service get discovered and protected. In his free time, he plays volleyball and develops his own projects. Martin Phan is the Field CTO in North America for CloudCasa by Catalogic Software. With over 20+ years of experience in the software-industry, he takes pride in supporting, developing, implementing, and selling enterprise software and data protection solutions to help customer solve their backup and recovery challenges. KEY TAKE-AWAYS FROM THE TALK 1) Challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. 2) Advantages of cloud awareness and Kubernetes managed service awareness for application and data recovery and security 3) Examples of overcoming Container Storage Interface (CSI) deviations 4) Various recovery options available in the market today.

Overcoming challenges with protecting and migrating data in multi-cloud K8s e...

DoKC

Link: https://youtu.be/YVXEpcSclwY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK In a continuation of a talk given at DoK day at KubeCon EU 2022, join Dinesh Majrekar, Civo's CTO as they walk through their evaluation process of the CNCF Storage market. Civo offers managed Kubernetes clusters powered by K3s to customers around the world. We manage thousands of Virtual Machines and stateful customer data within multiple data centres across several continents. In late 2021, Civo had the opportunity to evaluate the CNCF storage landscape to move to a new technology stack. During the migration project, Civo evaluated Mayastor, Ondat, Ceph and Longhorn against the following metrics: Scalability Performance Ease of Support Attendants will see practical examples on how they could carry out their own similar evaluation and see some of the results of the Civo research project. BIO Dinesh is CTO at Civo. Having worked in the hosting industry for many years, Dinesh has a passion for creating solutions that operate at scale. This not only applies to the technology stack, but for nurturing engineers through their career.

Evaluating Cloud Native Storage Vendors - DoK Talks #147

DoKC

Link: https://youtu.be/qUW8LkxYayc https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK How do you make sure your Stateful Workloads remain available when your Kubernetes infrastructure updates? This talk will discuss different strategies of upgrading a Kubernetes cluster, and how you can manage risk for your workload. The talk will showcase demos of each upgrade strategy. BIO Peter is a Senior Software Engineer on GKE at Google. He works on improving Kubernetes for Stateful workloads. His main focus is on enhancing the Kubernetes ecosystem for high availability applications. KEY TAKE-AWAYS FROM THE TALK The mechanics of different upgrade strategies, when to apply a particular upgrade strategy depending on your Stateful workload and how to mitigate risk to your application’s availability.

Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...

DoKC

Link: https://youtu.be/AjvwG53yLMY https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Stateful workloads are the heart of any application, yet they remain confusing and complicated even to daily K8s practitioners. That’s why many organizations shy away from migrating their data - their prized possession - to the unfamiliar stateful realm of Kubernetes. After meeting with many organizations in the adoption phase, I discovered what works best, what to avoid, and how critical it is to gain confidence and the right knowledge in order to successfully adopt stateful workloads. In this talk I will demonstrate how to optimally adopt Kubernetes and stateful workloads in a few steps, based on what I’ve learned from observing dozens of different adoption journeys. If you are taking your first steps in data on K8s or contemplating where to start - this talk is for you! BIO - A Developer turned Solution Architect. - Working at Komodor, a startup building the first K8s-native troubleshooting platform. - Love everything in infrastructure: storage, networks & security - from 70’s era mainframes to cloud-native. - All about “plan well, sleep well”. KEY TAKE-AWAYS FROM THE TALK - Understand how critical stateful workloads are for any system, and that the key challenges to migrating it to Kubernetes are knowledge and confidence. - How to build the foundational knowledge required to overcome adoption challenges by creating a learning path for individuals and teams. - How to gain confidence to run stateful workloads on Kubernetes with support from the community (and yourself!)

We will Dok You! - The journey to adopt stateful workloads on k8s

DoKC

Link: https://youtu.be/Pi5ueyl_1jU https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK During my first talk for DoK community I want to walk you through the world of NoSQL database MongoDB and Kubernetes Operators - Community Edition, Enterprise Edition (MongoDB and Ops Manager on K8s), and Atlas operator, highlight the most important capabilities, talk about use cases and challenges, the theory will be mixed with a live demos! BIO I'm a SRE / NoSQL / DevOps professional. I hold CKA, CKAD, CKS, also I’m MongoDB Certified DBA and MongoDB Champion. I have experience with multiple cloud providers, Kubernetes, different types of K8s operators (Strimzi, RabbitMQ Cluster Operator), but especially MongoDB K8s Operator. I also work with KEDA. Since 2017, I have been a speaker at MongoDB conferences all around the world (USA, China, Europe). KEY TAKE-AWAYS FROM THE TALK I would like to share the best practices of running NoSQL database - MongoDB on Kubernetes also I want to show how to manage Atlas (MongoDB cloud) via K8s operator https://www.mongodb.com/developer/community-champions/arkadiusz-borucki/

Mastering MongoDB on Kubernetes, the power of operators

DoKC

Link: https://youtu.be/KUipuM3UJF4 https://go.dok.community/slack https://dok.community/ DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Kubernetes comes with a lot of useful features like Volumes and StatefulSets, which make running stateful workloads simple. Interestingly, when combined with the right tools, these features can make Kubernetes very valuable for developers wanting to run massive production databases in development! This is exactly what was seen at "Extendi". The developers at Extendi deal with a large amount of data in their production Kubernetes clusters. But when developing locally, they didn't have an easy way of replicating this data. This replication was needed because it allowed developers to test new features instantaneously without worrying if they would work as expected when pushed to production. But replicating a 100Gb+ production database for development wasn't turning out to be an easy task! This is where leveraging Kubernetes + remote development environments came to the rescue. Running data on Kubernetes turned out to be way faster than any of the traditional approaches because of Kubernetes' ability to handle stateful workloads exceptionally well. And since Extendi already used Kubernetes in production - the setup process was fairly simple. This talk will cover practical steps on how leveraging Kubernetes based development environments allowed dev teams at Extendi to run production data on Kubernetes during development using features like Volume Snapshots, having a huge positive impact on developer productivity. ----- Arsh is a Developer Experience Engineer at Okteto. He is an active contributor to the upstream Kubernetes project and was awarded the Kubernetes Contributor Award for his contributions in 2021. Arsh has written blogs and spoken about different topics in the cloud-native ecosystem at various conferences before, including KubeCon + CloudNativeCon + Open Source Summit China 2021. He has also been on the Kubernetes Release Team since the 1.23 release. He also serves as the New Contributor Ambassador for the Documentation Special Interest Group of the Kubernetes project and continuously mentors new folks in the community. Previously, he worked at VMware and was an active contributor to other CNCF projects, including cert-manager and Kyverno. ----- Lapo is a Software Engineer currently leading the development team of a Social Listening and Audience Intelligence platform. He started coding at the early age of 14 and since he turned his passion into a real job, he has always been looking for boosting his knowledge by constantly researching for newer and newer technologies. Active on Ruby Open Source projects ----- Ramiro Berrelleza is one of the founders of Okteto. He has spent most of his career (and his free time) building cloud services and developer tools. Before starting Okteto, Ramiro was an Architect at Atlassian and a Software Engineer at Microsoft Azure.

Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...

DoKC

More from DoKC (20)

Distributed Vector Databases - What, Why, and How

Is It Safe? Security Hardening for Databases Using Kubernetes Operators

Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery

Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...

The State of Stateful on Kubernetes

Colocating Data Workloads and Web Services on Kubernetes to Improve Resource ...

Make Your Kafka Cluster Production-Ready

Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...

The Kubernetes Native Database

ING Data Services hosted on ICHP DoK Amsterdam 2023

Implementing data and databases on K8s within the Dutch government

StatefulSets in K8s - DoK Talks #154

Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...

Analytics with Apache Superset and ClickHouse - DoK Talks #151

Overcoming challenges with protecting and migrating data in multi-cloud K8s e...

Evaluating Cloud Native Storage Vendors - DoK Talks #147

Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...

We will Dok You! - The journey to adopt stateful workloads on k8s

Mastering MongoDB on Kubernetes, the power of operators

Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...

Recently uploaded

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Victor Rentea

In this keynote, Asanka Abeysinghe, CTO,WSO2 will explore the shift towards platformless technology ecosystems and their importance in driving digital adaptability and innovation. We will discuss strategies for leveraging decentralized architectures and integrating diverse technologies, with a focus on building resilient, flexible, and future-ready IT infrastructures. We will also highlight WSO2's roadmap, emphasizing our commitment to supporting this transformative journey with our evolving product suite.

Platformless Horizons for Digital Adaptability

WSO2

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

DBX First Quarter 2024 Investor Presentation

Dropbox

Key topics covered: - Real-world examples of Choreo's comprehensive coverage from application design and deployment, security, scaling, and monitoring - Running different types of workloads, such as web applications, APIs, microservices, integrations, and tasks at scale, and wire them together to deliver seamless omnichannel digital experiences - How Choreo improves the developer experience by eliminating repetition, silos, and redundancy through enhanced discoverability and self-serviceability

Choreo: Empowering the Future of Enterprise Software Engineering

WSO2

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

Design and Development of a Provenance Capture Platform for Data Science

Paolo Missier

Understanding the FAA Part 107 License ..

Christopher Logan Kennedy

Dubai, known for its towering skyscrapers, luxurious lifestyle, and relentless pursuit of innovation, often finds itself in the global spotlight. However, amidst the glitz and glamour, the emirate faces its own set of challenges, including the occasional threat of flooding. In recent years, Dubai has experienced sporadic but significant floods, disrupting normalcy and posing unique challenges to its infrastructure. Among the critical nodes in this bustling metropolis is the Dubai International Airport, a vital hub connecting the world. This article delves into the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Orbitshub

Key topics covered: - Understanding the basics of IAM and its significance in the modern enterprise. IAM in a platformless environment - Tackling real-world issues like prioritizing frictionless yet secure user access, securing high-value APIs, integrating to business, compliance, and adapting to cloud native environments with scalable solutions - Practical demonstrations of how WSO2 products can be instrumental in deploying efficient IAM solutions - Preparing for upcoming trends and innovations in identity management

Navigating Identity and Access Management in the Modern Enterprise

WSO2

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

Simplifying Mobile A11y Presentation.pptx

MarkSteadman7

Discover the innovative features and strategic vision that keep WSO2 an industry leader. Explore the exciting 2024 roadmap of WSO2 API management, showcasing innovations, unified APIM/APK control plane, natural language API interaction, and cloud native agility. Discover how open source solutions, microservices architecture, and cloud native technologies unlock seamless API management in today's dynamic landscapes. Leave with a clear blueprint to revolutionize your API journey and achieve industry success!

WSO2's API Vision: Unifying Control, Empowering Developers

WSO2

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

Explore the latest trends and insights on JavaScript usage with Pixlogix's informative blog. Discover key statistics and facts about JavaScript's role in web development, its popularity among developers, and its impact on modern websites. Stay updated with the evolving landscape of JavaScript frameworks and libraries, and learn how they're shaping the future of web development. Gain valuable insights to enhance your JavaScript skills and stay ahead in the digital realm.

JavaScript Usage Statistics 2024 - The Ultimate Guide

Pixlogix Infotech

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

How to Check CNIC Information Online with Pakdata cf

danishmna97

Elevate Developer Efficiency & build GenAI Application with Amazon Q

Bhuvaneswari Subramani

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Six Myths about Ontologies: The Basics of Formal Ontology

johnbeverley2021

Recently uploaded (20)

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Platformless Horizons for Digital Adaptability

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

DBX First Quarter 2024 Investor Presentation

Choreo: Empowering the Future of Enterprise Software Engineering

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Design and Development of a Provenance Capture Platform for Data Science

Understanding the FAA Part 107 License ..

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Navigating Identity and Access Management in the Modern Enterprise

Corporate and higher education May webinar.pptx

Simplifying Mobile A11y Presentation.pptx

WSO2's API Vision: Unifying Control, Empowering Developers

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

JavaScript Usage Statistics 2024 - The Ultimate Guide

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

How to Check CNIC Information Online with Pakdata cf

Elevate Developer Efficiency & build GenAI Application with Amazon Q

Why Teams call analytics are critical to your entire business

Six Myths about Ontologies: The Basics of Formal Ontology

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud

2. Sagi Volkov Distinguished Architect Lightbits Labs Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud November 6, 2023 | Chicago, Illinois

3. Agenda ● Intro to NVMe/TCP ● Why NVMe/TCP? ● Some performance comparison (and how I took them). ● Questions

4. NVMe/TCP - history lesson… ▪ NVM Express™ (NVMe) standard released in March 2011 ̶ Architecture, command set, and queueing interface for PCIe SSDs • Optimized for direct attached NVM PCIe® SSDs • The goal was a single interface that is scalable from client to enterprise ▪ NVMe™ over Fabrics (NVMe-oF™) standard released in June 2016 ̶ Extends the architecture, command set, and queueing interface of NVMe to fabric attached storage ̶ Two initial transport binding specifications • RDMA (Infiniband, RoCE, iWARP) • Fibre Channel ▪ TCP transport binding specification (NVMe™/TCP) specified for NVMe-oF in November of 2018

5. NVMe/TCP - more history… ▪ Also in 2018, Lightbits Labs introduced the first storage solution that was designed and built for NVMe/TCP ▪ Added to the Linux kernel in 4.10 ▪ Part of all major Linux enterprise distros. ▪ Today: Lightbits, Ceph, Dell, NetApp, Pure Storage and more.

6. NVMe-oF & NVMe/TCP - Protocol Layering

7. Why NVMe/TCP (or why do we need another NVMe-oF transport?) ▪ PCIe® ̶ Great for direct attached NVMe SSDs ̶ Does not scale well to large topologies ▪ RDMA (Infiniband, RoCE, iWARP) and Fibre Channel ̶ Provides a high degree of scalability, but requires special networks and hardware ▪ TCP ̶ Ubiquitous (does not require special networks or hardware) ̶ Scalable allowing large scale deployments and operation over long distances ̶ Can provide performance (throughput and latency) that is comparable to direct attached NVMe SSDs

8. Things to consider: ● Remember that you can use NVMe/TCP, wherever you have TCP available. Simple as that. ● Instance in the cloud usually have higher bandwidth for network than for native storage from the cloud provider. ● Network resources are billed differently depending on your cloud provider (ingress/egress)

9. Performance comparison (spec) ▪ Why PostgreSQL? ▪ Why Kubernetes? ▪ Why sysbench? https://github.com/akopytov/sysbench ▪ Why Sherlock? https://github.com/sagyvolkov/sherlock

10. Performance comparison (spec) Availability Zone VPC 18.75 Gbps R6in.8xlarge AWS Cloud I4i.x8large VPC Subnet I4i.x8large I4i.x8large 18.75 Gbps 18.75 Gbps VPC Subnet Pod: Sherlock (sysbench) cpu=0 memory=2Gi PVC Pod: Sherlock (sysbench) cpu=0 memory=2Gi PVC Pod: Sherlock Stats cpu=0.1 memory=64Mi

11. Performance comparison (spec) Sysbench test type Threads Lightbits io2 Block Express 75r25w 40 4,116,162 1,255,658 read-only 40 5,655,487 1,798,263 write-only 40 13,778,113 5,131,169 75r25w 75 5,256,168 1,895,664 read-only 75 7,193,175 2,835,951 write-only 75 14,983,822 6,236,878 75r25w 90 5,350,728 1,990,224 read-only 90 7,309,818 2,952,594 write-only 90 15,099,258 6,952,314 ● 1 worker - R6in.8x ● 3 PostgreSQL pods in the worker ● Each pod with Request/Limit of cpu=12 and memory=2Gi. ● Each pod with 300GB PVC. ● Each databases, 400 tables of 1M records.

12. Performance comparison (numbers)

13. Questions (and feedback)?

14. Thanks https://www.lightbitslabs.com My twitter: @clusterguru Sherlock: https://github.com/sagyvolkov/sherlock

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud

Recommended

Recommended

More Related Content

Similar to Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud

Similar to Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud (20)

More from DoKC

More from DoKC (20)

Recently uploaded

Recently uploaded (20)

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud