Eliminating the Double Write Problem in Apache Kafka Using the Outbox Pattern with Rafael Roman

•

0 likes•1,061 views

HostedbyConfluent

In distributed systems, it is common to encounter the double write problem, where the same data is written to multiple locations, leading to data inconsistencies. Apache Kafka is no exception to this, and these inconsistencies can cause significant issues in applications that rely on Kafka for data processing and storage. One effective solution to this problem is the outbox pattern, which involves storing all writes in a separate outbox table before they are replicated to the main data store. This allows the system to ensure that each write is only performed once, eliminating the risk of double writes. In this talk, we will discuss the double write problem in Apache Kafka and how the outbox pattern can be implemented to solve it. We will also demonstrate the use of the outbox pattern in a sample Kafka application and show how it can be used to ensure data consistency and integrity in distributed systems. Attendees will come away with a thorough understanding of the double write problem and how to use the outbox pattern to solve it in Apache Kafka and other distributed systems.

Rafael Roman
Principal Engineer

Double write
problem?

Everything will ultimately
fail.
Michael Nygard

This message is forever lost

Wrap in transaction

Wrap in transaction

Wrap in transaction

Thank you
Rafael Roman - Principal Engineer

More Related Content

More from HostedbyConfluent

"Discover how default configurations might impact ingestion times, especially when dealing with large files. We'll explore a real-world scenario with a 20,000,000+ line file, assessing metrics and exploring the bottleneck in the default setup. Understand the intricacies of batch size calculations and how to optimize them based on your unique data characteristics. Walk away with actionable insights as we showcase a practical example, turning a 7-hour ingestion process into a mere 30 minutes for over 30,000,000 records in a Kafka topic. Uncover metrics, configurations, and best practices to elevate the performance of your Kafka Connect CSV source connectors. Don't miss this opportunity to optimize your data pipeline and ensure smooth, efficient data flow."

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

HostedbyConfluent

"In order to meet the current and ever-increasing demand for near-zero RPO/RTO systems, a focus on resiliency is critical. While Kafka offers built-in resiliency features, a perfect blend of client and cluster resiliency is necessary in order to achieve a highly resilient Kafka client application. At Fidelity Investments, Kafka is used for a variety of event streaming needs such as core brokerage trading platforms, log aggregation, communication platforms, and data migrations. In this lightening talk, we will discuss the governance framework that has enabled producers and consumers to achieve their SLAs during unprecedented failure scenarios. We will highlight how we automated resiliency tests through chaos engineering and tightly integrated observability dashboards for Kafka clients to analyze and optimize client configurations. And finally, we will summarize the chaos test suite and the ""test, test and test"" mantra that are helping Fidelity Investments reach its goal of a future with zero down-time."

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

HostedbyConfluent

"There are various strategies for securely connecting to Kafka clusters between different networks or over the public internet. Many cloud providers even offer endpoints that privately route traffic between networks and are not exposed to the internet. But, depending on your network setup and how you are running Kafka, these options ... might not be an option! In this session, we’ll discuss how you can use SSH bastions or a self managed PrivateLink endpoint to establish connectivity to your Kafka clusters without exposing brokers directly to the internet. We explain the required network configuration, and show how we at Materialize have contributed to librdkafka to simplify these scenarios and avoid fragile workarounds."

Navigating Private Network Connectivity Options for Kafka Clusters

Navigating Private Network Connectivity Options for Kafka Clusters

Navigating Private Network Connectivity Options for Kafka Clusters

HostedbyConfluent

"In my talk, we will examine all the stages of building our self-service Streaming Data Platform based on Apache Flink and Kafka Connect, from the selection of a solution for stateful streaming data processing, right up to the successful design of a robust self-service platform, covering the challenges that we’ve met. I will share our experience in providing non-Java developers with a company-wide self-service solution, which allows them to quickly and easily develop their streaming data pipelines. Additionally, I will highlight specific business use cases that would not have been implemented without our platform.0 characters0 characters"

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

HostedbyConfluent

"Almost everyone has heard about large language models, and tens of millions of people have tried out OpenAI ChatGPT and Google Bard. However, the intricate architecture and underlying mathematics driving these remarkable systems remain elusive to many. LLM's are fascinating - so let's grab a drink and find out how these systems are built and dive deep into their inner workings. In the length of time it to enjoy a round of drinks, you'll understand the inner workings of these models. We'll take our first sip of word vectors, enjoy the refreshing taste of the transformer, and drain a glass understanding how these models are trained on phenomenally large quantities of data. Large language models for your streaming application - explained with a little maths and a lot of pub stories"

Explaining How Real-Time GenAI Works in a Noisy Pub

Explaining How Real-Time GenAI Works in a Noisy Pub

Explaining How Real-Time GenAI Works in a Noisy Pub

HostedbyConfluent

"Monitoring is a fundamental operation when running Kafka and Kafka applications in production. There are numerous metrics available when using Kafka, however the sheer number is overwhelming, making it challenging to know where to start and how to properly utilise them. This session will introduce you to some of the key metrics that should be monitored and best practices in fine tuning your monitoring. We will delve into which metrics are the indicators for cluster’s availability and performance and are the most helpful when debugging client applications."

TL;DR Kafka Metrics | Kafka Summit London

TL;DR Kafka Metrics | Kafka Summit London

TL;DR Kafka Metrics | Kafka Summit London

HostedbyConfluent

Kafka Streams relies on state restoration for maintaining standby tasks as failure recovery mechanism as well as for restoring the state after rebalance scenarios. When you are scaling up or down your application instances, it is necessary to know the current state of the restoration process for each active and standby task in order to prevent a long restoration process as much as possible. During this presentation, you will get an understanding of how KIP-869 provides valuable information about the current active task restoration after a rebalance and KIP-988 opens a window to the continuous process of standby restoration. When you encounter a situation in which you need to choose whether or not to scale up or down your application instances, both KIPs will be an invaluable ally for you.

A Window Into Your Kafka Streams Tasks | KSL

A Window Into Your Kafka Streams Tasks | KSL

A Window Into Your Kafka Streams Tasks | KSL

HostedbyConfluent

"In this talk, we will dive into the world of Kafka producer configs and explore how to understand and optimize them for better performance. We will cover the different types of configs, their impact on performance, and how to tune them to achieve the best results. Whether you're new to Kafka or a seasoned pro, this session will provide valuable insights and practical tips for improving your Kafka producer performance. - Introduction to Kafka producer internal and workflow - Understanding the producer configs like linger.ms, batch.size, buffer.memory and their impact on performance - Learning about producer configs like max.block.ms, delivery.timeout.ms, request.timeout.ms and retries to make producer more resilient. - Discuss configs like enable.idempotence, max.in.flight.requests.per.connection and transaction related configs to achieve delivery guarantees. - Q&A session with attendees to address specific questions and concerns."

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

HostedbyConfluent

"Data contracts are one of the hottest topics in the data management community. A data contract is a formal agreement between a data producer and its consumers, aimed at reducing data downtime and improving data quality. Schemas are an important part of data contracts, but they are not the only relevant element. In this talk, we’ll: 1. see why data contracts are so important but also difficult to implement; 2. identify the characteristics of a well-designed data contract: discuss the anatomy of a data contract, its main elements and, how to formally describe them; 3. show how to manage the lifecycle of a data contract leveraging Confluent Platform's services."

Data Contracts Management: Schema Registry and Beyond

Data Contracts Management: Schema Registry and Beyond

Data Contracts Management: Schema Registry and Beyond

HostedbyConfluent

"In the realm of stateful stream processing, Apache Flink has emerged as a powerful and versatile platform. However, the conventional SQL-based approach often limits the full potential of Flink applications. We will delve into the benefits of adopting a code-first approach, which provides developers with greater control over application logic, facilitates complex transformations, and enables more efficient handling of state and time. We will also discuss how the code-first approach can lead to more maintainable and testable code, ultimately improving the overall quality of your Flink applications. Whether you're a seasoned Flink developer or just starting your journey, this talk will provide valuable insights into how a code-first approach can revolutionize your stream processing applications."

Code-First Approach: Crafting Efficient Flink Apps

Code-First Approach: Crafting Efficient Flink Apps

Code-First Approach: Crafting Efficient Flink Apps

HostedbyConfluent

"Change Data Capture (CDC) has become a commodity in data engineering, much in part due to the ever-rising success of Debezium [1]. But is that all there is? In this lightning talk, we’ll outline the current state of the CDC ecosystem, and understand why adopting a Debezium alternative is still a hard sell. If you’ve ever wondered what else is out there, but can’t keep up with the sprawling of new tools in the ecosystem; we’ll wrap it up for you! [1] https://debezium.io/"

Debezium vs. the World: An Overview of the CDC Ecosystem

Debezium vs. the World: An Overview of the CDC Ecosystem

Debezium vs. the World: An Overview of the CDC Ecosystem

HostedbyConfluent

"Separation of compute and storage has become the de-facto standard in the data industry for batch processing. The addition of tiered storage to open source Apache Kafka is the first step in bringing true separation of compute and storage to the streaming world. In this talk, we'll discuss in technical detail how to take the concept of tiered storage to its logical extreme by building an Apache Kafka protocol compatible system that has zero local disks. Eliminating all local disks in the system requires not only separating storage from compute, but also separating data from metadata. This is a monumental task that requires reimagining Kafka's architecture from the ground up, but the benefits are worth it. This approach enables a stateless, elastic, and serverless deployment model that minimizes operational overhead and also drives inter-zone networking costs to almost zero."

Beyond Tiered Storage: Serverless Kafka with No Local Disks

Beyond Tiered Storage: Serverless Kafka with No Local Disks

Beyond Tiered Storage: Serverless Kafka with No Local Disks

HostedbyConfluent

"Regular performance testing is one of the pillars of Kafka Streams’ reliability and efficiency. Beyond ensuring dependable releases, regular performance testing supports engineers in new feature development with the ability to easily test the performance impact of their features, compare different approaches, etc. In this session, Alex and John share their experience from developing, using, and maintaining a performance testing framework for Kafka Streams that has prevented multiple performance regressions over the last 5 years. They cover guiding principles and architecture, how to ensure statistical significance and stability of results, and how to automate regression detection for actionable notifications. This talk sheds light on how Apache Kafka is able to foster a vibrant open-source community while maintaining a high performance bar across many years and releases. It also empowers performance-minded engineers to avoid common pitfalls and bring high-quality performance testing to their own systems."

Automating Speed: A Proven Approach to Preventing Performance Regressions in ...

Automating Speed: A Proven Approach to Preventing Performance Regressions in ...

Automating Speed: A Proven Approach to Preventing Performance Regressions in ...

HostedbyConfluent

"The energy transition has brought with it a wave of new challenges for the high-voltage grid. One of the central problems is the volatility of renewable energies. A future-proof control center must have enough flexibility to meet the demands of the energy market in real time. Existing industrial systems cannot meet the requirements of modern network operators. Our projects aim to solve these challenges with sophisticated event-based architectures, with Apache Kafka at its core. These architectures need to solve complex requirements like switching high critical grid assets and integrate hundreds of thousands of sensors. We used a couple of event- based patterns like SAGAs, Event Sourcing and more to overcome those challenges. Additionally, we had to integrate the OT systems in this modern tech stack. (IEC, OPCUA) Some projects are already in shadow operation and are expected to steer first grid parts in 2025."

How to Build an Event-based Control Center for the Electrical Grid

How to Build an Event-based Control Center for the Electrical Grid

How to Build an Event-based Control Center for the Electrical Grid

HostedbyConfluent

"Apache Kafka is the cool kid on the block when it comes to distributed streaming platforms. It's gained popularity in cloud environments due to its scalability, fault tolerance, and real-time data processing capabilities. As more organizations move their data processing and analysis workflows to the cloud, Kafka has become an essential part of their architecture. But, let's face it, running Kafka in the cloud can be a bit of a headache. It requires careful planning and execution to ensure optimal performance, reliability, and cost-effectiveness. In this talk, we're going to introduce you to a little something called ""showbacks"". Showbacks are a way of giving users feedback on the resources they're consuming and the associated costs. It's like telling your friends to keep their hands out of the cookie jar before they eat all the cookies. It also works with developers and partitions. We'll even give you some real-world examples of how New Relic has successfully implemented showbacks to achieve cost savings and operational efficiency in our Kafka deployments like auto-scaling, Availability Zone awareness, or client tunning. We promise to keep it fun, technical, and full of practical insights and actionable recommendations to help you achieve optimal Kafka performance and cost-effectiveness in the cloud."

Keep Your Kafka Cloud Costs in Check with Showbacks

Keep Your Kafka Cloud Costs in Check with Showbacks

Keep Your Kafka Cloud Costs in Check with Showbacks

HostedbyConfluent

"The Norwegian health care sectors success is dependent on the security and availability of enormous amounts of critical data. The state of affairs is that data does not flow. At Norsk helsenett we have recently started using Kafka to secure this flow. Our critical data is your critical data, and a matter of life and death. This I how we do that: - Redundancy to us starts with the number 3, creating the infrastructure and using it to manage ""always available"" services and data, working towards zero downtime and data loss. - Security is paramount, how did we implement mTLS? - How do we monitor and support usage of data we cannot look at using the benefits of .Net and SignalR in Blazor together with Kafka."

When Securing Access to Data is About Life and Death

When Securing Access to Data is About Life and Death

When Securing Access to Data is About Life and Death

HostedbyConfluent

"Invidi ad decisioning engine needs semi-realtime feedback on the performance of the ad campaigns it runs. In the heart of this feedback loop there is a service that aggregates 1B+ daily ad tracking events and serves campaign performance time series to the ad decisioning engine over http. Recently we successfully rewrote it as a pure Kafka Streams application with all data being stored in Kafka and served via Interactive Queries. The experience was surprisingly not straightforward and we had to trade off some of the simplicity of our processing topology to increase scalability and lower resource consumption. In this talk we plan to go over the system architecture and share the issues we faced and how we solved them. Here are some highlights: - The distribution of our aggregation keys is very skewed, so early repartitioning resulted in poor scalability. To mitigate this issue we used a scatter-gather approach avoiding repartitioning and combining results in IQ. To minimize memory consumption we had to combine the above approach with pre-aggregating events before re-partitioning in a lambda-architecture style. - Due to multiple stores sharing buffer memory we had to resort to manually deleting entries in our live windowed stores to avoid premature flushing of the aggregates due to cache thrashing. - We had to implement our own in-memory windowed store to increase IQ performance We hope that our findings will be helpful to a wider audience. We also plan to file and fix the issues we discovered in the near future."

Aggregating Ad Events with Kafka Streams and Interactive Queries at Invidi

Aggregating Ad Events with Kafka Streams and Interactive Queries at Invidi

Aggregating Ad Events with Kafka Streams and Interactive Queries at Invidi

HostedbyConfluent

"Consumer scaling is a crucial element for many Apache Kafka users. Who doesn’t want to save money by efficiently managing their resources, shutting down unnecessary instances when there is no traffic, quickly scaling up during peak hours and while doing all of that - avoiding annoying and often unnecessary rebalancing. To achieve all of this you need to understand how consumer assignment works, how nodes are affected by data load and what are common causes of rebalancing. But most importantly - what assignors to choose based on your use case and what metrics to use to measure your data load. You wonder how we know what are good and bad practices? At Aiven, we've seen firsthand both successful and not-so-great approaches to consumer scaling and rebalancing. The insights we're sharing with you come directly from our experience working on many projects with Apache Kafka. We’ll discuss metrics that are essential for understanding data load and deciding when to scale. We'll cover a variety of approaches you can take - from commonly used lag exporters, to Knative scalers that are based on concurrent requests and finally insights from our own experience developing a speed lag predictor that goes beyond the basics by calculating the velocity of data load changes. We’ll highlight advantages and disadvantages of each approach and when you should use it. Next, we'll look at various assignors that are available and guide you on how to choose the most suitable one for your scenario. We'll pay special attention to the challenges faced by stateful applications and the potential pitfalls of frequent scaling, such as overloaded brokers. Armed with this knowledge, you’ll have what is needed to build scalable systems, minimize downtime and save costs when working with Apache Kafka. Let's make your Kafka experience as smooth and efficient as possible!"

Mastering Kafka Consumer Distribution: A Guide to Efficient Scaling and Resou...

Mastering Kafka Consumer Distribution: A Guide to Efficient Scaling and Resou...

Mastering Kafka Consumer Distribution: A Guide to Efficient Scaling and Resou...

HostedbyConfluent

"The Apache Flink community is working on a significant milestone with the planned release of Flink 2.0, marking a major evolution since the inception of Flink 1.0 in 2016. In this insightful talk, we will delve into the key enhancements and transformations slated for the 2.0 version, offering a comprehensive overview for users and developers eager to embrace the next frontier of stream and batch processing. The talk will commence by exploring some of the core philosophies of Flink, emphasizing the unification of batch and streaming processing. We'll dissect the roadmap's commitment to a seamless blending of batch and streaming applications. Flink 2.0 isn't just about features; it's also a reimagining of Flink's architecture. We'll delve into the disaggregated state management approach, leveraging distributed file systems. This evolution is geared towards better load balancing and improved efficiency in cloud-native architectures. APIs play a pivotal role in Flink's evolution, and Flink 2.0 is no exception. The talk will outline plans to retire deprecated APIs, overhaul the configuration layer, new abstractions and of course other plans that live in the community. Join us on this exploration of Flink 2.0, where we'll unravel the future of stream and batch processing and showcase how these advancements will shape the landscape of real-time data analytics."

Flink 2.0: Navigating the Future of Unified Stream and Batch Processing

Flink 2.0: Navigating the Future of Unified Stream and Batch Processing

Flink 2.0: Navigating the Future of Unified Stream and Batch Processing

HostedbyConfluent

"In the realm of real time streaming application, Kafka is commonly chosen as the datastore. This demands the use of faster and expensive storage at scale. However, the need for high-performance storage volume often translates to steep costs, hindering its wider adoption. To tackle this challenge, we have integrated the tiered storage feature introduced in KIP-405 in our Strimzi-operated Kafka. With tiered storage and our custom remote plugin, we provide a cost-effective solution allowing data to be stored in a cheaper storage, and thus enabling extended retention periods and allow efficiently backfilling of past data. This innovation allows our streaming applications to benefit without the strain of hefty storage expenses. In this session, we'll familiarize audience on how we use Kafka to build an end-to-end streaming applications while also touching the behind-the-scene aspect of how we were able to reduce the cost to make this affordable for wider adoption. We will share our experience on integrating the tiered storage feature with Strimzi Kafka operator and Strimzi operated Kafka clusters. Moreover, we will share our journey of optimizing performance for our remote storage manager implementation and the valuable insights gained along the way."

Leveraging Tiered Storage in Strimzi-Operated Kafka for Cost-Effective Stream...

Leveraging Tiered Storage in Strimzi-Operated Kafka for Cost-Effective Stream...

Leveraging Tiered Storage in Strimzi-Operated Kafka for Cost-Effective Stream...

HostedbyConfluent

More from HostedbyConfluent (20)

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

Navigating Private Network Connectivity Options for Kafka Clusters

Navigating Private Network Connectivity Options for Kafka Clusters

Navigating Private Network Connectivity Options for Kafka Clusters

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

Explaining How Real-Time GenAI Works in a Noisy Pub

Explaining How Real-Time GenAI Works in a Noisy Pub

Explaining How Real-Time GenAI Works in a Noisy Pub

TL;DR Kafka Metrics | Kafka Summit London

TL;DR Kafka Metrics | Kafka Summit London

TL;DR Kafka Metrics | Kafka Summit London

A Window Into Your Kafka Streams Tasks | KSL

A Window Into Your Kafka Streams Tasks | KSL

A Window Into Your Kafka Streams Tasks | KSL

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

Data Contracts Management: Schema Registry and Beyond

Data Contracts Management: Schema Registry and Beyond

Data Contracts Management: Schema Registry and Beyond

Code-First Approach: Crafting Efficient Flink Apps

Code-First Approach: Crafting Efficient Flink Apps

Code-First Approach: Crafting Efficient Flink Apps

Debezium vs. the World: An Overview of the CDC Ecosystem

Debezium vs. the World: An Overview of the CDC Ecosystem

Debezium vs. the World: An Overview of the CDC Ecosystem

Beyond Tiered Storage: Serverless Kafka with No Local Disks

Beyond Tiered Storage: Serverless Kafka with No Local Disks

Beyond Tiered Storage: Serverless Kafka with No Local Disks

Automating Speed: A Proven Approach to Preventing Performance Regressions in ...

Automating Speed: A Proven Approach to Preventing Performance Regressions in ...

Automating Speed: A Proven Approach to Preventing Performance Regressions in ...

How to Build an Event-based Control Center for the Electrical Grid

How to Build an Event-based Control Center for the Electrical Grid

How to Build an Event-based Control Center for the Electrical Grid

Keep Your Kafka Cloud Costs in Check with Showbacks

Keep Your Kafka Cloud Costs in Check with Showbacks

Keep Your Kafka Cloud Costs in Check with Showbacks

When Securing Access to Data is About Life and Death

When Securing Access to Data is About Life and Death

When Securing Access to Data is About Life and Death

Aggregating Ad Events with Kafka Streams and Interactive Queries at Invidi

Aggregating Ad Events with Kafka Streams and Interactive Queries at Invidi

Aggregating Ad Events with Kafka Streams and Interactive Queries at Invidi

Mastering Kafka Consumer Distribution: A Guide to Efficient Scaling and Resou...

Mastering Kafka Consumer Distribution: A Guide to Efficient Scaling and Resou...

Mastering Kafka Consumer Distribution: A Guide to Efficient Scaling and Resou...

Flink 2.0: Navigating the Future of Unified Stream and Batch Processing

Flink 2.0: Navigating the Future of Unified Stream and Batch Processing

Flink 2.0: Navigating the Future of Unified Stream and Batch Processing

Leveraging Tiered Storage in Strimzi-Operated Kafka for Cost-Effective Stream...

Leveraging Tiered Storage in Strimzi-Operated Kafka for Cost-Effective Stream...

Leveraging Tiered Storage in Strimzi-Operated Kafka for Cost-Effective Stream...

Recently uploaded

Angeliki Cooney has spent over twenty years at the forefront of the life sciences industry, working out of Wynantskill, NY. She is highly regarded for her dedication to advancing the development and accessibility of innovative treatments for chronic diseases, rare disorders, and cancer. Her professional journey has centered on strategic consulting for biopharmaceutical companies, facilitating digital transformation, enhancing omnichannel engagement, and refining strategic commercial practices. Angeliki's innovative contributions include pioneering several software-as-a-service (SaaS) products for the life sciences sector, earning her three patents. As the Senior Vice President of Life Sciences at Avenga, Angeliki orchestrated the firm's strategic entry into the U.S. market. Avenga, a renowned digital engineering and consulting firm, partners with significant entities in the pharmaceutical and biotechnology fields. Her leadership was instrumental in expanding Avenga's client base and establishing its presence in the competitive U.S. market.

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Angeliki Cooney

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Passkeys: Developing APIs to enable passwordless authentication Cody Salas, Sr Developer Advocate | Solutions Architect - Yubico Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Accelerating FinTech Innovation: Unleashing API Economy and GenAI Vasa Krishnan, Chief Technology Officer - FinResults Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

AXA XL - Insurer Innovation Award Americas 2024

AXA XL - Insurer Innovation Award Americas 2024

AXA XL - Insurer Innovation Award Americas 2024

The Digital Insurer

Manulife - Insurer Transformation Award 2024

Manulife - Insurer Transformation Award 2024

Manulife - Insurer Transformation Award 2024

The Digital Insurer

The microservices honeymoon is over. When starting a new project or revamping a legacy monolith, teams started looking for alternatives to microservices. The Modular Monolith, or 'Modulith', is an architecture that reaps the benefits of (vertical) functional decoupling without the high costs associated with separate deployments. This talk will delve into the advantages and challenges of this progressive architecture, beginning with exploring the concept of a 'module', its internal structure, public API, and inter-module communication patterns. Supported by spring-modulith, the talk provides practical guidance on addressing the main challenges of a Modultith Architecture: finding and guarding module boundaries, data decoupling, and integration module-testing. You should not miss this talk if you are a software architect or tech lead seeking practical, scalable solutions. About the author With two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Axa Assurance Maroc - Insurer Innovation Award 2024

Axa Assurance Maroc - Insurer Innovation Award 2024

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Dubai, often portrayed as a shimmering oasis in the desert, faces its own set of challenges, including the occasional threat of flooding. Despite its reputation for opulence and modernity, the emirate is not immune to the forces of nature. In recent years, Dubai has experienced sporadic but significant floods, testing the resilience of its infrastructure and communities. Among the critical lifelines in this bustling metropolis is the Dubai International Airport, a bustling hub that connects the city to the world. This article explores the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Rustici Software

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

In the thrilling conclusion to 2023, ransomware groups had a banner year, really outdoing themselves in the "make everyone's life miserable" department. LockBit 3.0 took gold in the hacking olympics, followed by the plucky upstarts Clop and ALPHV/BlackCat. Apparently, 48% of organizations were feeling left out and decided to get in on the cyber attack action. Business services won the "most likely to get digitally mugged" award, with education and retail nipping at their heels. Hackers expanded their repertoire beyond boring old encryption to the much more exciting world of extortion. The US, UK and Canada took top honors in the "countries most likely to pay up" category. Bitcoins were the currency of choice for discerning hackers, because who doesn't love untraceable money?

Ransomware_Q4_2023. The report. [EN].pdf

Ransomware_Q4_2023. The report. [EN].pdf

Ransomware_Q4_2023. The report. [EN].pdf

Overkill Security

Recently uploaded (20)

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

AXA XL - Insurer Innovation Award Americas 2024

AXA XL - Insurer Innovation Award Americas 2024

AXA XL - Insurer Innovation Award Americas 2024

Manulife - Insurer Transformation Award 2024

Manulife - Insurer Transformation Award 2024

Manulife - Insurer Transformation Award 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Axa Assurance Maroc - Insurer Innovation Award 2024

Axa Assurance Maroc - Insurer Innovation Award 2024

Axa Assurance Maroc - Insurer Innovation Award 2024

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Ransomware_Q4_2023. The report. [EN].pdf

Ransomware_Q4_2023. The report. [EN].pdf

Ransomware_Q4_2023. The report. [EN].pdf

Eliminating the Double Write Problem in Apache Kafka Using the Outbox Pattern with Rafael Roman

1.

2. Rafael Roman Principal Engineer

3. Double write problem?

4.

5. Everything will ultimately fail. Michael Nygard

6.

7. This message is forever lost

8. Wrap in transaction

9. Wrap in transaction

10. Wrap in transaction

11.

12.

13. Thank you Rafael Roman - Principal Engineer