Blockchain and Kafka - A Modern Love Story | Suhavi Sandhu, Guidewire Software

•

0 likes•656 views

After familiarizing myself with blockchain over the past couple years, I noticed a few caveats and setbacks that make blockchain a ‘not-so-ideal’ solution to record tracking in a P2P network. It’s not as scalable and it’s quite slow in a busy network. Enter Apache Kafka, with its high performance and immutable logging. In this talk, I want to explore the relationship between Blockchain and Kafka and demonstrate how the two technologies can benefit from each other. If you’re interested in the future of blockchain and love Kafka, this is definitely up your alley.

Technology

Blockchain
and Kafka
A Modern Love Story
Suhavi Sandhu

Agenda
What is a DApp?
01
Popular Use Cases for DApps
02
Advantages/Disadvantages of Blockchain
03
Kafka and Blockchain
04
Demo
05
Final Remarks
06

What is a
Decentralized
Application?
A program that runs on ”blockchain" or ”P2P” networks.
They combine a smart contract and a frontend user interface
to give users more control over their online data. Using a
currency like Ether, DApps are transforming the way the
Internet works

Popular
Use
Cases for
DApps
Supply chain and logistics
Voting mechanisms
Digital identity
Cryptocurrency exchange
Secure sharing of medical data

Cons of
Blockchain
Slow
High energy consumption
Technical complexity
Pros of
Blockchain
Immutability
Transparency
Efficiency

Kafka can solve some of our
problems
REALTIME
TRANSACTIONS
Instead of waiting for 5
minutes, instantaneous
response
HIGH
THROUGHPUT
Ability to ingest
monstrous amount of
data per second
MESSAGE
RETENTION
An immutable commit
log to replay all
transactions

Kafka and
Blockchain
An experiment
Kafka shares a lot of parallels with distributed
ledgers and what they aim to achieve. So why not
try to combine the two?
Kafka
UI Blockchain
Payment
Gateway

A Simple
Approach
Two Topics
A topic for unverified transactions and a topic for
verified transactions. The goal is to keep them in sync.
Producer
t1 t2
Consumer

Final Remarks
Kafka and Blockchain
Has a chance of working in a trustful environment.
Still a long way to go until decentralized apps
become the norm but until then we need to keep
experimenting.

Thanks for listening!
SuhaviSandhu
gucci-ninja
Suhavi Sandhu

What's hot

Today’s products - devices, software and services - are well instrumented to permit users, vendors and service providers to gather maximum insight into how they are used, when they need repair and many other operational insights. Ensuring that products can rapidly adapt to a constantly changing environment and changing customer needs requires that the events they generate are analyzed continuously and in context. Insights can be synthesized from many sources in context - geospatial and proximity, trajectory and even predicted future states.Customers, vendors and service providers need to analyze, learn, and predict directly from streaming events because data volumes are huge and automated responses must often be delivered in milliseconds. To achieve insights quickly, we need to build models on-the-fly whose predictions are accurate and in sync with the real world, often to support automation. Many insights depend on analyzing the joint evolution of data sources whose behavior is correlated in time or space.In this talk we present Swim, an Apache 2.0 licensed platform for continuous intelligence applications. Swim builds a fluid model of data sources and their changing relationships in real-time - Swim applications analyze, learn and predict directly from event data. Swim applications integrate with Apache Kafka for event streaming. Developers need nothing more than Java skills. Swim deploys native or in containers on k8s, with the same code in each instance. Instances link to build an application layer mesh that facilitates distribution and massive scale without sacrificing consistency. We will present several continuous intelligence applications in use today that depend on real-time analysis, learning and prediction to power automation and deliver responses that are in sync with the real-world. We will show how easy it is to build, deploy and run distributed, highly available event streaming applications that analyze data from hundreds of millions of sources - petabytes per day. The architecture is intuitively appealing and blazingly fast.

Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...

HostedbyConfluent

Kafka Vienna Meetup 020719

Patrik Kleindl

Modern data processing applications built on Kafka and InfluxDB deliver the performance, reliability, and flexibility that customers need for robust real-time data pipeline solutions. As the saying goes, the pipeline is greater than the sum of its Kafka and InfluxDB parts. In this session, Russ Savage, Director of Product Management at InfluxData will discuss basic concepts of integrating Kafka and InfluxDB while highlighting how companies are creating fault-tolerant, scalable and fast data pipelines with the power of InfluxDB and Kafka.

Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...

HostedbyConfluent

Stream me to the Cloud (and back) with Confluent & MongoDB

confluent

Keynote: Jay Kreps, Confluent | Kafka ♥ Cloud | Kafka Summit 2020

confluent

Elastically Scaling Kafka Using Confluent

confluent

Apache Kafka is an open source event streaming platform. It is often used to complement or even replace existing middleware to integrate applications and build microservice architectures. Apache Kafka is already used in various projects in almost every bigger company today. Understood, battled-tested, highly scalable, reliable, real-time. Blockchain is a different story. This technology is a lot in the news, especially related to cryptocurrencies like Bitcoin. But what is the added value for software architectures? Is blockchain just hype and adds complexity? Or will it be used by everybody in the future, like a web browser or mobile app today? And how is it related to an integration architecture and event streaming platform? This session explores use cases for blockchains and discusses different alternatives such as Hyperledger, Ethereum and a Kafka-native tamper-proof blockchain implementation. Different architectures are discussed to understand when blockchain really adds value and how it can be combined with the Apache Kafka ecosystem to integrate blockchain with the rest of the enterprise architecture to build a highly scalable and reliable event streaming infrastructure. Speakers: Kai Waehner, Technology Evangelist, Confluent Stephen Reed, CTO, Co-Founder, AiB

Building a Secure, Tamper-Proof & Scalable Blockchain on Top of Apache Kafka ...

confluent

The data that organizations are required to analyze in order to make informed decisions is growing at an unprecedented rate. Companies have to capture the window of opportunity and become not just data driven, but event driven. In this talk, we will talk around addressing these issues and look into ways to bridge the on-premise kafka deployments with GCP stack for different use cases and personas. This will be followed by architecture examples on How do you deploy kafka and integrate with the rest of the GCP stack.

Hybrid Streaming Analytics for Apache Kafka Users | Firat Tekiner, Google

HostedbyConfluent

Government Track Welcome Address

HostedbyConfluent

Apache Druid is a high-performance distributed analytics store for modern analytics applications. It supports ingesting millions of events per second and sub-second query processing. Druid supports various types of data sources for ingestion, including Apache Kafka. You can immediately query on stream events once they get ingested into Druid. Since Kafka provides scalable and robust data delivery while Druid supports advanced complex analysis on streams, Kafka and Druid are widely used together for BI and operational analytics use cases, which require interactivity, scalability, real-time, and performance. This talk is based on our real-world experiences building out streaming analytics stacks powering production use cases across many industries.

Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...

HostedbyConfluent

APIs have become ubiquitous as a way of exposing the capabilities of the enterprise both internally and externally. However, are APIs alone enough? There is a strong resurgence in interest in asynchronous communication and event driven architecture. Applications want to receive events immediately so they can respond in real time, and furthermore they also want the benefit of being decoupled from the availability and performance characteristics of the systems providing that data. However, whilst the way that APIs are socialized, exposed, versioned etc. is well matured in the form of API management technology. We are now on the cusp of seeing first class support for event endpoint management to provide the same sophistication for discovering, exposing and consuming events.

Should we manage events like APIs? | Kim Clark, IBM

HostedbyConfluent

Google Cloud Next ’19 Speakers: Gaetan Castelein, Confluent Product Marketing Kir Titievsky, Google Product Management Confluent Cloud for Apache Kafka® was a session conducted at Google Cloud Next ’19 on the topic of how Confluent and Google are partnering to give you a complete event-streaming platform that extends Kafka with essential capabilities for developers and enterprises. Confluent is available as a fully managed, first class service on GCP, or can be deployed on-premises on Google Cloud Services Platform. Developers can deploy Confluent Cloud™ in minutes right from the Google Cloud Console to start building event-driven applications. Enterprises can build hybrid cloud streaming solutions with a common platform that spans from on-premises to GCP, streaming data to GCP to leverage best-of-breed services such as BigQuery and TensorFlow. Review this presentation to learn about Confluent and GCP services, and see how you can get started in just minutes with no upfront commitment.

Confluent Cloud for Apache Kafka® | Google Cloud Next ’19

confluent

Van Oord, a 150 year old family owned business, build windmill parks in the sea, lay cables on sea surface, dredging, as well as infrastructure (Dike, etc) operates world-wide, often facilitating self-owned specialized vessels. A well-known prestigious project is the creation of the palm island at the coast of Dubai. Data Management in Van Oord is still in its infancy. The current operation is based on bilateral data exchange, without an Enterprise Service Bus or mayor data warehouse infrastructure. In 2020 Van Oord started a PoC with Confluent Kafka, executing a wide range of uses cases and requirements, followed by the formal program implementing a sustainable data platform. Data owners are publishing an information product, i.e. a set of Kafka topics to communicate change (a la CDC) and topics for sharing state of a data source (Kafka tables). The information product owner is responsible for granting access, assuring data quality, data linage and governance. The set of all information products forms the enterprise data model. This talk outlines why Van Oord requires data governance and enterprise architecture models integrated with Confluent Kafka, and demo how an open-source based data governance tool is integrated with Confluent Kafka to fulfil these requirements.

What does an event mean? Manage the meaning of your data! | Andreas Wombacher...

HostedbyConfluent

(Gaurav Goyal + Dharmesh Panchmatia, Cisco Systems) Kafka Summit SF 2018 Cisco e-commerce platform is a custom-built mission-critical platform which accounts for $40+ billion of Cisco’s revenue annually. It’s a suite of 35 different applications and 300+ services that powers product configuration, pricing, quoting and order booking across all Cisco product lines including hardware, software, services and subscriptions. It’s a B2B platform used by the Cisco sales team, partners and direct customers, serving 140,000 unique users across the globe. In order to improve customer experience and business agility, Cisco decided to transition the platform to cloud-native technologies, MongoDB, Elasticsearch and Kafka. In this session, we will share details around: -Kafka architecture -How we are experiencing significant resiliency advantages, zero-downtime deployment and improved performance -How we’ve implemented Kafka to pass data to 20+ downstream applications, removing point-to-point integrations, batch jobs and standardizing the handshake -How are we using Kafka for pushing data for machine learning and analytics use cases -Best practices and lessons learned

Cisco’s E-Commerce Transformation Using Kafka

confluent

NAV handles a lot of applications for our benefits(sickness, unemployment etc.). We are rewriting the systems handling these applications to a stream based approach, placing Kafka-streams in the centre of our architecture. This talk will go through our approach to building these systems. We are mostly using Kotling as our programming language, and are running our applications on Kubernetes. We use Change-data-capture to extract data from our legacy systems, to ease the pain of migration. By treating our data, and our applications as streams we get a lot of benefits. We can rerun our applications with different rules, to simulate the effect of policy changes. We can do real time evaluation of applications, to improve the user experience. We also get better robustness, as we remove the runtime dependency to our core systems, and can receive applications even when other parts of the systems are down. The talk will also present the architecural rules we are implementing for NAV, using streams as the main communication pattern between organization units, and some learnings from our current work on a data platform to support Life is a stream of events.

Work is a Stream of Applications (Audun Strand, NAV) Kafka Summit London 2019

confluent

Organisations are becoming Event Driven based on streaming technologies and adopting Data Mesh and Event Mesh architectures. As this becomes pervasive, so do the challenges around runtime governance and lifecycle management. For example, do you know what streams exist, who is producing and consuming them? What is the effect of upstream changes? How is this information kept up to date, and how do people collaborate efficiently across distributed teams and environments? Ever wish you had a way to view and visualize graphically the relationships between schemas, topics and applications? In this talk we will show you how to do that and get more value from your Kafka Streaming infrastructure using an Event Portal - an API portal specialized for event streams and publish/subscribe patterns. Join us to see how you can discover event streams from your Kafka clusters, import them to a catalog to see alongside other enterprise event streams and leverage code gen capabilities to ease development.

Mesh-ing around with Streams across the Enterprise | Phil Scanlon, Solace

HostedbyConfluent

Data in motion has changed both the scale and scope of data and analytics - enabling organizations to capture more information and use it more effectively. But to get the most value from it - you need to know what’s there, make it risk aware, and take action on it. In this session, you’ll learn how to leverage modern ML-augmented data management solutions to automatically find, identify, and classify sensitive data across Spark, Databricks, and beyond - and how to apply policies for compliance and risk mitigation to get the most value from our data.

Scalable Data Management for Kafka and Beyond | Dan Rice, BigID

HostedbyConfluent

ASRC Federal created the Mission Operator Assist (MOA) tool to extend human capabilities through AI/ML for NOAA. MOA ingests system log data from on-orbit satellite constellations and applies machine learning to greatly improve real-time situational awareness. MOA uses a collection of tools, including Kafka for multi-subscriber communications, all hosted through AWS Cloud Services and Kubernetes Containers for microservices. Like many traditional on-premises systems, satellite ground station operations are undergoing a renaissance as they increasingly become enabled by cloud. During this session, the audience will learn about the satellite communications chain, and best practices and lessons learned in creating a data pipeline with Kafka for high throughput and scalability while displaying high quality situational awareness to mission operators. We will discuss our goals centered around establishing event-driven streaming for satellite logs so our machine learning becomes real-time and supporting a multi-subscriber approach for various Kafka topics. Listeners will also learn how a multi-subscriber approach using Kafka, helped us auto scale logstash based on how many messages are in the queue and other microservices.

Kafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC Federal

HostedbyConfluent

Guru Sattanathan, Confluent, Senior Solutions Engineer Enterprise Integration technologies (aka Middleware) are the key enablers when it comes to Real-time data flows or Event Driven Architecture. Starting from real-time payments, e-commerce, travel booking systems, etc, everything is powered by a middleware underneath. It did transform a lot of things but with caveats! Are ESB’s & MQ’s enough for today’s integration needs? Do you know their technical debts? If you are someone looking at integrating your applications or an Integration Architect this session is for you. It's time to refresh yourself and see how organizations are building integrations today. In this session, we will go in this order: -Recap on Enterprise Integration technologies -What are the key flaws & What needs improvement? -What is Apache Kafka? -Rethinking Integration using Apache Kafka https://www.meetup.com/KafkaMelbourne/events/280590162/

Death of the dumb pipes: Using Apache Kafka® for Integration projects

HostedbyConfluent

Netflix will spend roughly 19B+ on content in 2021 over 100 countries to entertain over 200M subscribers. With this scale of investment in internationalized productions, it is essential to intelligently invest and track the movement of cash, in near-real-time, from pre-production to launch. This involves complex near real time financial calculations, data movement, consistent data views of transactional data of the present and the cash predictions of the future. It needs exactly once processing semantics and idempotency while guaranteeing a low latency serving layer. We leverage kafka to build a scalable and fault tolerance event-based processing engine to provide the right SLA guarantees. We also embrace event-driven data materialization to provide low latency lookups at scale. The aim of this talk is to provide an insight into how we built such a scalable system while embracing a full blown Kappa architecture, with Kafka at the heart of it all.

Money Heist - A Stream Processing Original! | Meha Pandey and Shengze Yu, Net...

HostedbyConfluent

What's hot (20)

Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...

Kafka Vienna Meetup 020719

Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...

Stream me to the Cloud (and back) with Confluent & MongoDB

Keynote: Jay Kreps, Confluent | Kafka ♥ Cloud | Kafka Summit 2020

Elastically Scaling Kafka Using Confluent

Building a Secure, Tamper-Proof & Scalable Blockchain on Top of Apache Kafka ...

Hybrid Streaming Analytics for Apache Kafka Users | Firat Tekiner, Google

Government Track Welcome Address

Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...

Should we manage events like APIs? | Kim Clark, IBM

Confluent Cloud for Apache Kafka® | Google Cloud Next ’19

What does an event mean? Manage the meaning of your data! | Andreas Wombacher...

Cisco’s E-Commerce Transformation Using Kafka

Work is a Stream of Applications (Audun Strand, NAV) Kafka Summit London 2019

Mesh-ing around with Streams across the Enterprise | Phil Scanlon, Solace

Scalable Data Management for Kafka and Beyond | Dan Rice, BigID

Kafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC Federal

Death of the dumb pipes: Using Apache Kafka® for Integration projects

Money Heist - A Stream Processing Original! | Meha Pandey and Shengze Yu, Net...

Similar to Blockchain and Kafka - A Modern Love Story | Suhavi Sandhu, Guidewire Software

Cloud computing and Software defined networking

saigandham1

Open Standards Enabling Digital Transformation

Solace

More info: https://cnfl.io/cloud-native-experience-for-kafka-in-cloud | Neha Narkhede is co-founder and CTO at Confluent, a company backing the popular Apache Kafka messaging system. Prior to founding Confluent, Neha led streams infrastructure at LinkedIn, where she was responsible for LinkedIn’s streaming infrastructure built on top of Apache Kafka and Apache Samza. She is one of the initial authors of Apache Kafka and a committer and PMC member on the project.

Neha Narkhede | Kafka Summit London 2019 Keynote | Event Streaming: Our Cloud...

confluent

Streaming Data and Stream Processing with Apache Kafka

confluent

API Design and WebSocket

Frank Greco

Etl is Dead; Long Live Streams

confluent

Apache Kafka vs. Traditional Middleware (Kai Waehner, Confluent) Frankfurt 20...

confluent

MQ, ETL and ESB middleware are often used as integration backbone between legacy applications, modern microservices and cloud services. This introduces several challenges and complexities like point-to-point integration or non-scalable architectures. This session discusses how to build a completely event-driven streaming platform leveraging Apache Kafka’s open source messaging, integration and streaming components to leverage distributed processing, fault-tolerance, rolling upgrades and the ability to reprocess events. Learn the differences between a event-driven streaming platform leveraging Apache Kafka and middleware like MQ, ETL and ESBs – including best practices and anti-patterns, but also how these concepts and tools complement each other in an enterprise architecture.

Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...

confluent

Learn the differences between an event-driven streaming platform and middleware like MQ, ETL and ESBs – including best practices and anti-patterns, but also how these concepts and tools complement each other in an enterprise architecture. Extract-Transform-Load (ETL) is still a widely-used pattern to move data between different systems via batch processing. Due to its challenges in today’s world where real time is the new standard, an Enterprise Service Bus (ESB) is used in many enterprises as integration backbone between any kind of microservice, legacy application or cloud service to move data via SOAP / REST Web Services or other technologies. Stream Processing is often added as its own component in the enterprise architecture for correlation of different events to implement contextual rules and stateful analytics. Using all these components introduces challenges and complexities in development and operations. This session discusses how teams in different industries solve these challenges by building a native streaming platform from the ground up instead of using ETL and ESB tools in their architecture. This allows to build and deploy independent, mission-critical streaming real time application and microservices. The architecture leverages distributed processing and fault-tolerance with fast failover, no-downtime rolling deployments and the ability to reprocess events, so you can recalculate output when your code changes. Integration and Stream Processing are still key functionality but can be realized in real time natively instead of using additional ETL, ESB or Stream Processing tools.

Apache Kafka vs. Integration Middleware (MQ, ETL, ESB)

Kai Wähner

1. Software-Defined Networks (SDN) is a new paradigm in network management that adds another layer (i.e., Network Operating System) to the architecture. Answer the following questions in the context of SDN with your reasoning. (a) Is it scalable? Why? (b) Is it less responsive? Why? (c) Does it create a single point of failure? Why? (d) Is it inherently less secure? Why? (e) Is it incrementally deployable? Why? 2.RED randomly drops packets when it experience congestion. The probability of drop increases as the average queue size increases. (a) Does it do a better job for uniform or bursty traffic? and why? (b) Does it drop packets from the head of the queue or from the tail of the queue? and why? (c) Does it make any difference; head/tail drop? and why? 3. Carefully read the short article OpenFlow: A Radical New Idea in Networking (http://queue.acm.org/detail.cfm?id=2305856), and answer the following questions. The author argues that the deployment of SDN in general and OpenFlow in specific towards network democratization is a crazy idea. Do you agree? If yes, how come SDN has been supported and being deployed by many networking vendors. If not, give one scenario that SDN could cause disruptions. NET WORKS 1 OpenFlow: A Radical New Idea in Networking An open standard that enables software-defined networking Thomas A. Limoncelli Computer networks have historically evolved box by box, with individual network elements occupying specific ecological niches as routers, switches, load balancers, NATs (network address translations), or firewalls. Software-defined networking proposes to overturn that ecology, turning the network as a whole into a platform and the individual network elements into programmable entities. The apps running on the network platform can optimize traffic flows to take the shortest path, just as the current distributed protocols do, but they can also optimize the network to maximize link utilization, create different reachability domains for different users, or make device mobility seamless. OpenFlow, an open standard that enables software-defined networking in IP networks, is a new network technology that will enable many new applications and new ways of managing networks. Here are three real, though somewhat fictionalized, applications: EXAMPLE 1: BANDWIDTH MANAGEMENT. A typical wide area network has 30 percent utilization; it must “reserve” bandwidth for “burst” times. Using OpenFlow, however, a system was developed in which internal application systems (consumers) that need bulk data transfer could use the spare bandwidth. Typical uses include daily replication of datasets, database backups, and the bulk transmission of logs. Consumers register the source, destination, and quantity of data to be transferred with a central service. The service does various calculations and sends the results to the routers so they know how to forward this bulk data when links are otherwise unused. Communication ...

1. Software-Defined Networks (SDN) is a new paradigm in network ma.docx

jackiewalcutt

Floodlight tutorial - Clemson / Georgia Tech

openflowhub

Carrier As A Service15102009

vnandlall

Jay Kreps is the CEO of Confluent, Inc., a company backing the popular Apache Kafka® messaging system. Prior to founding Confluent, he was formerly the lead architect for data infrastructure at LinkedIn. He is among the original authors of several open source projects including Project Voldemort (a key-value store). Apache Kafka (a distributed messaging system) and Apache Samza (a stream processing system).

Jay Kreps | Kafka Summit NYC 2019 Keynote (Events Everywhere) | CEO, Confluent

confluent

As cloud computing continues to dominate the technology landscape, this comprehensive slide deck from IPFS Things 2023 sheds light on the potential of a decentralized approach to cloud computing. While centralized cloud computing has facilitated the rapid growth of Web 2.0 and Web 3.0, it comes with considerable downsides such as the oligopoly of hyperscalers and an inverse correlation between developer experience and centralization. With over a billion internet-facing servers worldwide, major cloud providers represent less than 1% of global computing capacity. The key to unlocking the remaining potential lies in decentralizing cloud computing through innovative software solutions. Explore the essential components of a decentralized cloud computing platform, including: - Peer-to-peer networks that foster a collaborative and distributed ecosystem - Cloud features as protocols to enable seamless communication and interoperability - Serverless architecture for location-independent computing, similar to content-addressing - WebAssembly, a portable, secure, and interoperable technology that supports efficient storage and execution of code Learn about Taubyte, a company striving to build a decentralized implementation for cloud computing that can harness the untapped potential of global server capacity. By leveraging serverless models, WebAssembly, and content-addressing, Taubyte aims to create a scalable, secure, and accessible platform to power the next generation of internet applications. Unlock the potential of decentralized cloud computing and explore the building blocks of this transformative technology with this insightful slide deck from IPFS Things 2023. Join the movement towards a more equitable, efficient, and innovative future in cloud computing.

How to Build a Decentralized Cloud Computing Platform?

Samy Fodil

Gluecon - Kafka and the service mesh

Gwen (Chen) Shapira

Interledger Protocol Stack Deep Dive @ Boston Interledger Meetup

Interledger

SDN: an introduction

Luca Profico

IoT Story: From Edge to HDP

DataWorks Summit

Ipv4 Vs Ipv6

April Charlton

Big Data LDN 2017: The Rise Of The Streaming Platform

Matt Stubbs

Similar to Blockchain and Kafka - A Modern Love Story | Suhavi Sandhu, Guidewire Software (20)

Cloud computing and Software defined networking

Open Standards Enabling Digital Transformation

Neha Narkhede | Kafka Summit London 2019 Keynote | Event Streaming: Our Cloud...

Streaming Data and Stream Processing with Apache Kafka

API Design and WebSocket

Etl is Dead; Long Live Streams

Apache Kafka vs. Traditional Middleware (Kai Waehner, Confluent) Frankfurt 20...

Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...

Apache Kafka vs. Integration Middleware (MQ, ETL, ESB)

1. Software-Defined Networks (SDN) is a new paradigm in network ma.docx

Floodlight tutorial - Clemson / Georgia Tech

Carrier As A Service15102009

Jay Kreps | Kafka Summit NYC 2019 Keynote (Events Everywhere) | CEO, Confluent

How to Build a Decentralized Cloud Computing Platform?

Gluecon - Kafka and the service mesh

Interledger Protocol Stack Deep Dive @ Boston Interledger Meetup

SDN: an introduction

IoT Story: From Edge to HDP

Ipv4 Vs Ipv6

Big Data LDN 2017: The Rise Of The Streaming Platform

More from HostedbyConfluent

"In this talk, attendees will be provided with an introduction to Kafka Connect and the basics of Single Message Transforms (SMTs) and how they can be used to transform data streams in a simple and efficient way. SMTs are a powerful feature of Kafka Connect that allow custom logic to be applied to individual messages as they pass through the data pipeline. The session will explain how SMTs work, the types of transformations they can be used for, and how they can be applied in a modular and composable way. Further, the session will discuss where SMTs fit in with Kafka Connect and when they should be used. Examples will be provided of how SMTs can be used to solve common data integration challenges, such as data enrichment, filtering, and restructuring. Attendees will also learn about the limitations of SMTs and when it might be more appropriate to use other tools or frameworks. Additionally, an overview of the alternatives to SMTs, such as Kafka Streams and KSQL, will be provided. This will help attendees make an informed decision about which approach is best for their specific use case. Whether attendees are developers, data engineers, or data scientists, this talk will provide valuable insights into how Kafka Connect and SMTs can help streamline data processing workflows. Attendees will come away with a better understanding of how these tools work and how they can be used to solve common data integration challenges."

Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...

HostedbyConfluent

"While Apache Kafka lacks native support for topic renaming, there are scenarios where renaming topics becomes necessary. This presentation will delve into the utilization of MirrorMaker 2.0 as a solution for renaming Kafka topics. It will illustrate how MirrorMaker 2.0 can efficiently facilitate the migration of messages from the old topic to the new one and how Kafka Connect Metrics can be employed to monitor the mirroring progress. The discussion will encompass the complexity of renaming Kafka topics, addressing certain limitations, and exploring potential workarounds when using MirrorMaker 2.0 for this purpose. Despite not being originally designed for topic renaming, MirrorMaker 2.0 has a suitable solution for renaming Kafka topics. Blog Post : https://engineering.hellofresh.com/renaming-a-kafka-topic-d6ff3aaf3f03"

Renaming a Kafka Topic | Kafka Summit London

HostedbyConfluent

"Trendyol, Turkey's leading e-commerce company, is committed to positively impacting the lives of millions of customers. Our decision-making processes are entirely driven by data. As a data warehouse team, our primary goal is to provide accurate and up-to-date data, enabling the extraction of valuable business insights. We utilize the benefits provided by Kafka and Kafka Connect to facilitate the transfer of data from the source to our analytical environment. We recently transitioned our Kafka Connect clusters from on-premise VMs to Kubernetes. This shift was driven by our desire to effectively manage rapid growth(marked by a growing number of producers, consumers, and daily messages), ensuring proper monitoring and consistency. Consistency is crucial, especially in instances where we employ Single Message Transforms to manipulate records like filtering based on their keys or converting a JSON Object into a JSON string. Monitoring our cluster's health is key and we achieve this through Grafana dashboards and alerts generated through kube-state-metrics. Additionally, Kafka Connect's JMX metrics, coupled with NewRelic, are employed for comprehensive monitoring. The session will aim to explain our approach to NRT data ingestion, outlining the role of Kafka and Kafka Connect, our transition journey to K8s, and methods employed to monitor the health of our clusters."

Evolution of NRT Data Ingestion Pipeline at Trendyol

HostedbyConfluent

"Join our lightning talk to delve into the strategies vital for maintaining a resilient Kafka service. While proactive monitoring is key for issue prevention, failures will still occur. Rapid detection tools will enable you to identify and resolve problems before they impact end-users. This session explores the techniques employed by Kafka cloud providers for this detection, many of which are also applicable if you are managing independent Kafka clusters or applications. The talk focuses on health-checking, a powerful tool that encompasses an application and its monitoring to validate Kafka environment availability. The session navigates through Kafka health-check methods, sharing best practices, identifying common pitfalls, and highlighting the monitoring of critical performance metrics like throughput and latency for early issue detection. Attendees will gain valuable insights into the art of health-checking their Kafka environment, equipping them with the tools to identify and address issues before they escalate into critical problems. We invite all Kafka enthusiasts to join us in this talk to foster a deeper understanding of Kafka health-checking and ensure the continued smooth operation of your Kafka environment."

Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques

HostedbyConfluent

"Stream processing systems traditionally gave their users the choice between at least once processing and at most once processing: accepting duplicate data or missing data. But ideally we would provide exactly-once processing, where every event in the input data is represented exactly once in the output. Kafka provides a transaction API that enables exactly-once when using Kafka as your source and sink. But this API has turned out to not be well suited for use by high level streaming systems, requiring various work arounds to still provide transactional processing. In this talk, I’ll cover how the transaction API works, and how systems like Arroyo and Flink have used it to build exactly-once support, and how improvements to the transactional API will enable better end-to-end support for consistent stream processing."

Exactly-once Stream Processing with Arroyo and Kafka

HostedbyConfluent

"In this talk, we will explore the exciting world of IoT and computer vision by presenting a unique project: Fish Plays Pokemon. Using an ESP Eye camera connected to an ESP32 and other IoT devices, to monitor fish's movements in an aquarium. This project showcases the power of IoT and computer vision, demonstrating how even a fish can play a popular video game. We will discuss the challenges we faced during development, including real-time processing, IoT device integration, and Kafka message consumption. By the end of the talk, attendees will have a better understanding of how to combine IoT, computer vision, and the usage of a serverless cloud to create innovative projects. They will also learn how to integrate IoT devices with Kafka to simulate keyboard behavior, opening up endless possibilities for real-time interactions between the physical and digital worlds."

Fish Plays Pokemon | Kafka Summit London

HostedbyConfluent

Tiered Storage 101 | Kafla Summit London

HostedbyConfluent

"Real-time 24/7 monitoring and verification of massive data is challenging – even more so for the world’s second largest manufacturer of memory chips and semiconductors. Tolerance levels are incredibly small, any small defect needs to be identified and dealt with immediately. The goal of semiconductor manufacturing is to improve yield and minimize unnecessary work. However, even with real-time data collection, the data was not easy to manipulate by users and it took many days to enable stream processing requests – limiting its usefulness and value to the business. You’ll hear why SK hynix switched to Confluent and how we developed a self-service stream process portal on top of it. Now users have an easy-to-use service to manipulate the data they want. Results have been impressive, stream processing requests are available the same day – previously taking 5 days! We were also able to drive down costs by 10% as stream processing requests no longer require additional hardware. What you’ll take away from our talk: - What were the pain points in the previous environment - How we transitioned to Confluent without service downtime - Creating a self-service stream processing portal built on top of Connect and ksqlDB - Use case of stream process portal"

Building a Self-Service Stream Processing Portal: How And Why

HostedbyConfluent

"Discover how default configurations might impact ingestion times, especially when dealing with large files. We'll explore a real-world scenario with a 20,000,000+ line file, assessing metrics and exploring the bottleneck in the default setup. Understand the intricacies of batch size calculations and how to optimize them based on your unique data characteristics. Walk away with actionable insights as we showcase a practical example, turning a 7-hour ingestion process into a mere 30 minutes for over 30,000,000 records in a Kafka topic. Uncover metrics, configurations, and best practices to elevate the performance of your Kafka Connect CSV source connectors. Don't miss this opportunity to optimize your data pipeline and ensure smooth, efficient data flow."

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

HostedbyConfluent

"In order to meet the current and ever-increasing demand for near-zero RPO/RTO systems, a focus on resiliency is critical. While Kafka offers built-in resiliency features, a perfect blend of client and cluster resiliency is necessary in order to achieve a highly resilient Kafka client application. At Fidelity Investments, Kafka is used for a variety of event streaming needs such as core brokerage trading platforms, log aggregation, communication platforms, and data migrations. In this lightening talk, we will discuss the governance framework that has enabled producers and consumers to achieve their SLAs during unprecedented failure scenarios. We will highlight how we automated resiliency tests through chaos engineering and tightly integrated observability dashboards for Kafka clients to analyze and optimize client configurations. And finally, we will summarize the chaos test suite and the ""test, test and test"" mantra that are helping Fidelity Investments reach its goal of a future with zero down-time."

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

HostedbyConfluent

"There are various strategies for securely connecting to Kafka clusters between different networks or over the public internet. Many cloud providers even offer endpoints that privately route traffic between networks and are not exposed to the internet. But, depending on your network setup and how you are running Kafka, these options ... might not be an option! In this session, we’ll discuss how you can use SSH bastions or a self managed PrivateLink endpoint to establish connectivity to your Kafka clusters without exposing brokers directly to the internet. We explain the required network configuration, and show how we at Materialize have contributed to librdkafka to simplify these scenarios and avoid fragile workarounds."

Navigating Private Network Connectivity Options for Kafka Clusters

HostedbyConfluent

"In my talk, we will examine all the stages of building our self-service Streaming Data Platform based on Apache Flink and Kafka Connect, from the selection of a solution for stateful streaming data processing, right up to the successful design of a robust self-service platform, covering the challenges that we’ve met. I will share our experience in providing non-Java developers with a company-wide self-service solution, which allows them to quickly and easily develop their streaming data pipelines. Additionally, I will highlight specific business use cases that would not have been implemented without our platform.0 characters0 characters"

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

HostedbyConfluent

"Almost everyone has heard about large language models, and tens of millions of people have tried out OpenAI ChatGPT and Google Bard. However, the intricate architecture and underlying mathematics driving these remarkable systems remain elusive to many. LLM's are fascinating - so let's grab a drink and find out how these systems are built and dive deep into their inner workings. In the length of time it to enjoy a round of drinks, you'll understand the inner workings of these models. We'll take our first sip of word vectors, enjoy the refreshing taste of the transformer, and drain a glass understanding how these models are trained on phenomenally large quantities of data. Large language models for your streaming application - explained with a little maths and a lot of pub stories"

Explaining How Real-Time GenAI Works in a Noisy Pub

HostedbyConfluent

"Monitoring is a fundamental operation when running Kafka and Kafka applications in production. There are numerous metrics available when using Kafka, however the sheer number is overwhelming, making it challenging to know where to start and how to properly utilise them. This session will introduce you to some of the key metrics that should be monitored and best practices in fine tuning your monitoring. We will delve into which metrics are the indicators for cluster’s availability and performance and are the most helpful when debugging client applications."

TL;DR Kafka Metrics | Kafka Summit London

HostedbyConfluent

Kafka Streams relies on state restoration for maintaining standby tasks as failure recovery mechanism as well as for restoring the state after rebalance scenarios. When you are scaling up or down your application instances, it is necessary to know the current state of the restoration process for each active and standby task in order to prevent a long restoration process as much as possible. During this presentation, you will get an understanding of how KIP-869 provides valuable information about the current active task restoration after a rebalance and KIP-988 opens a window to the continuous process of standby restoration. When you encounter a situation in which you need to choose whether or not to scale up or down your application instances, both KIPs will be an invaluable ally for you.

A Window Into Your Kafka Streams Tasks | KSL

HostedbyConfluent

"In this talk, we will dive into the world of Kafka producer configs and explore how to understand and optimize them for better performance. We will cover the different types of configs, their impact on performance, and how to tune them to achieve the best results. Whether you're new to Kafka or a seasoned pro, this session will provide valuable insights and practical tips for improving your Kafka producer performance. - Introduction to Kafka producer internal and workflow - Understanding the producer configs like linger.ms, batch.size, buffer.memory and their impact on performance - Learning about producer configs like max.block.ms, delivery.timeout.ms, request.timeout.ms and retries to make producer more resilient. - Discuss configs like enable.idempotence, max.in.flight.requests.per.connection and transaction related configs to achieve delivery guarantees. - Q&A session with attendees to address specific questions and concerns."

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

HostedbyConfluent

"Data contracts are one of the hottest topics in the data management community. A data contract is a formal agreement between a data producer and its consumers, aimed at reducing data downtime and improving data quality. Schemas are an important part of data contracts, but they are not the only relevant element. In this talk, we’ll: 1. see why data contracts are so important but also difficult to implement; 2. identify the characteristics of a well-designed data contract: discuss the anatomy of a data contract, its main elements and, how to formally describe them; 3. show how to manage the lifecycle of a data contract leveraging Confluent Platform's services."

Data Contracts Management: Schema Registry and Beyond

HostedbyConfluent

"In the realm of stateful stream processing, Apache Flink has emerged as a powerful and versatile platform. However, the conventional SQL-based approach often limits the full potential of Flink applications. We will delve into the benefits of adopting a code-first approach, which provides developers with greater control over application logic, facilitates complex transformations, and enables more efficient handling of state and time. We will also discuss how the code-first approach can lead to more maintainable and testable code, ultimately improving the overall quality of your Flink applications. Whether you're a seasoned Flink developer or just starting your journey, this talk will provide valuable insights into how a code-first approach can revolutionize your stream processing applications."

Code-First Approach: Crafting Efficient Flink Apps

HostedbyConfluent

"Change Data Capture (CDC) has become a commodity in data engineering, much in part due to the ever-rising success of Debezium [1]. But is that all there is? In this lightning talk, we’ll outline the current state of the CDC ecosystem, and understand why adopting a Debezium alternative is still a hard sell. If you’ve ever wondered what else is out there, but can’t keep up with the sprawling of new tools in the ecosystem; we’ll wrap it up for you! [1] https://debezium.io/"

Debezium vs. the World: An Overview of the CDC Ecosystem

HostedbyConfluent

"Separation of compute and storage has become the de-facto standard in the data industry for batch processing. The addition of tiered storage to open source Apache Kafka is the first step in bringing true separation of compute and storage to the streaming world. In this talk, we'll discuss in technical detail how to take the concept of tiered storage to its logical extreme by building an Apache Kafka protocol compatible system that has zero local disks. Eliminating all local disks in the system requires not only separating storage from compute, but also separating data from metadata. This is a monumental task that requires reimagining Kafka's architecture from the ground up, but the benefits are worth it. This approach enables a stateless, elastic, and serverless deployment model that minimizes operational overhead and also drives inter-zone networking costs to almost zero."

Beyond Tiered Storage: Serverless Kafka with No Local Disks

HostedbyConfluent

More from HostedbyConfluent (20)

Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...

Renaming a Kafka Topic | Kafka Summit London

Evolution of NRT Data Ingestion Pipeline at Trendyol

Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques

Exactly-once Stream Processing with Arroyo and Kafka

Fish Plays Pokemon | Kafka Summit London

Tiered Storage 101 | Kafla Summit London

Building a Self-Service Stream Processing Portal: How And Why

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

Navigating Private Network Connectivity Options for Kafka Clusters

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

Explaining How Real-Time GenAI Works in a Noisy Pub

TL;DR Kafka Metrics | Kafka Summit London

A Window Into Your Kafka Streams Tasks | KSL

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

Data Contracts Management: Schema Registry and Beyond

Code-First Approach: Crafting Efficient Flink Apps

Debezium vs. the World: An Overview of the CDC Ecosystem

Beyond Tiered Storage: Serverless Kafka with No Local Disks

Recently uploaded

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Accelerating FinTech Innovation: Unleashing API Economy and GenAI Vasa Krishnan, Chief Technology Officer - FinResults Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

apidays

Passkeys: Developing APIs to enable passwordless authentication Cody Salas, Sr Developer Advocate | Solutions Architect - Yubico Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

apidays

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Keynote 2: APIs in 2030: The Risk of Technological Sleepwalk Paolo Malinverno, Growth Advisor - The Business of Technology Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

apidays

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the deployment of external web forms using Jotform for Bonterra Impact Management. This solution can be customized to your organization’s needs and deployed to support the common use cases below: - Intake and consent - Assessments - Surveys - Applications - Program registration Interested in deploying web form automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Jeffrey Haguewood

Discover the innovative features and strategic vision that keep WSO2 an industry leader. Explore the exciting 2024 roadmap of WSO2 API management, showcasing innovations, unified APIM/APK control plane, natural language API interaction, and cloud native agility. Discover how open source solutions, microservices architecture, and cloud native technologies unlock seamless API management in today's dynamic landscapes. Leave with a clear blueprint to revolutionize your API journey and achieve industry success!

WSO2's API Vision: Unifying Control, Empowering Developers

WSO2

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

Retrieval augmented generation (RAG) is the most popular style of large language model application to emerge from 2023. The most basic style of RAG works by vectorizing your data and injecting it into a vector database like Milvus for retrieval to augment the text output generated by an LLM. This is just the beginning. One of the ways that we can extend RAG, and extend AI, is through multilingual use cases. Typical RAG is done in English using embedding models that are trained in English. In this talk, we’ll explore how RAG could work in languages other than English. We’ll explore French, Chinese, and Polish.

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Zilliz

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

ICT role in 21st century education and its challenges

rafiqahmad00786416

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

MINDCTI Revenue Release Quarter One 2024

MIND CTI

Architecting Cloud Native Applications

WSO2

Recently uploaded (20)

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Why Teams call analytics are critical to your entire business

Boost Fertility New Invention Ups Success Rates.pdf

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

WSO2's API Vision: Unifying Control, Empowering Developers

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

Artificial Intelligence Chap.5 : Uncertainty

Introduction to Multilingual Retrieval Augmented Generation (RAG)

presentation ICT roal in 21st century education

ICT role in 21st century education and its challenges

How to Troubleshoot Apps for the Modern Connected Worker

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

MINDCTI Revenue Release Quarter One 2024

Architecting Cloud Native Applications

Blockchain and Kafka - A Modern Love Story | Suhavi Sandhu, Guidewire Software

1. Blockchain and Kafka A Modern Love Story Suhavi Sandhu

2. Agenda What is a DApp? 01 Popular Use Cases for DApps 02 Advantages/Disadvantages of Blockchain 03 Kafka and Blockchain 04 Demo 05 Final Remarks 06

3. What is a Decentralized Application? A program that runs on ”blockchain" or ”P2P” networks. They combine a smart contract and a frontend user interface to give users more control over their online data. Using a currency like Ether, DApps are transforming the way the Internet works

4. Popular Use Cases for DApps Supply chain and logistics Voting mechanisms Digital identity Cryptocurrency exchange Secure sharing of medical data

5. Cons of Blockchain Slow High energy consumption Technical complexity Pros of Blockchain Immutability Transparency Efficiency

6. Kafka can solve some of our problems REALTIME TRANSACTIONS Instead of waiting for 5 minutes, instantaneous response HIGH THROUGHPUT Ability to ingest monstrous amount of data per second MESSAGE RETENTION An immutable commit log to replay all transactions

7. Kafka and Blockchain An experiment Kafka shares a lot of parallels with distributed ledgers and what they aim to achieve. So why not try to combine the two? Kafka UI Blockchain Payment Gateway

8. A Simple Approach Two Topics A topic for unverified transactions and a topic for verified transactions. The goal is to keep them in sync. Producer t1 t2 Consumer

9. DEMO

10. Final Remarks Kafka and Blockchain Has a chance of working in a trustful environment. Still a long way to go until decentralized apps become the norm but until then we need to keep experimenting.

11. Thanks for listening! SuhaviSandhu gucci-ninja Suhavi Sandhu

Blockchain and Kafka - A Modern Love Story | Suhavi Sandhu, Guidewire Software

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Blockchain and Kafka - A Modern Love Story | Suhavi Sandhu, Guidewire Software

Similar to Blockchain and Kafka - A Modern Love Story | Suhavi Sandhu, Guidewire Software (20)

More from HostedbyConfluent

More from HostedbyConfluent (20)

Recently uploaded

Recently uploaded (20)

Blockchain and Kafka - A Modern Love Story | Suhavi Sandhu, Guidewire Software