Transforming The Customer Experience With Real-Time Insights
•
1 like•124 views
Streaming data pipelines allow almost any organization to deliver higher value, faster, to customers. Delivering on the real-time element of the customer experience, however, necessitated a major overhaul to HPE's architecture. In this Gartner Catalyst 2018 session, Jeff Dutton, Data Platform Architect for HPE, will share his journey developing a new architecture to deliver real-time insights from billions of IOT sensors, trillions of metrics, and petabytes of data.
Report
Share
Report
Share
Download to read offline
Recommended
Executive Briefing: What Is Fast Data And Why Is It Important
[About This Webinar]
Streaming data systems, so called Fast Data, promise accelerated access to information, leading to new innovations and competitive advantages. These systems, however, aren’t just faster versions of Big Data; they force architecture changes to meet new demands for reliability and dynamic scalability, more like microservices.
This means new challenges for your organization. Whereas a batch job might run for hours, a stream processing application might run for weeks or months. This raises the bar for making these systems resilient against traffic spikes, hardware and network failures, and so forth. The good news is that there is a strong history of facing these demands in the world of microservices.
In this webinar by Dr. Dean Wampler, VP of Fast Data Architecture at Lightbend, Inc., we will cut through the buzz around Fast Data and explore how to successfully exploit this new opportunity for innovation in how your organization leverages data. Specifically, Dean will review:
* The business justification for transitioning from batch-oriented big data to stream-oriented fast data
* The architectural and organizational changes that streaming systems require to meet their higher demands for reliability, resiliency, dynamic scalability, etc.
* How some of these requirements can be met by leveraging what your organization already knows about microservice architectures
What's The Role Of Machine Learning In Fast Data And Streaming Applications?
Designed for busy Architects and Managers, this Lightbend Webinar features Dr. Emre Velipasaoglu, Principal Data Scientist at Lightbend, who explains what ML is really all about, the ideal use cases for ML, and how getting it right can benefit your streaming and Fast Data application architectures.
[INFOGRAPHIC] Event-driven Business: How to Handle the Flow of Event Data
Get key research stats on why CIOs are turning to streaming data for a competitive advantage.
Improving Veteran benefit services through efficient data streaming | Robert ...
Having information on our Veterans and Veteran services is not an issue. The VA (VETERANS AFFAIRS) currently has petabytes of data on Veterans, the services they receive and the level of benefit to which those services align. The challenge we are aiming to solve through Confluent is the ability to publish eventing data in a manner consumable by various products - be they in–house applications or VA mobile apps - to rapidly notify a Veteran or service provider about key information concerning a claim or services received. Such information could include a change in status of a claim, requests for additional documentation to support a claim or appeal or what information should be shared when calling into the call center or what level of benefits are due when checking in at a hospital. This technology will enable the VA to provide accurate, real-time information on a claim, appeal or rating for our Veterans.
Data reply sneak peek: real time decision engines
Events happen constantly in every business: a purchase in an online shop, a credit limit is hit, the mobile internet plan has been exhausted, users interact with a website. Events rule the business world. So why would you react to them hours or days later? Real-Time Decision Engines enable a variety of use cases, driving new products, increasing user experience, reducing costs and risks by reacting instantly to business events.
From personalized instantaneous marketing campaigns to reacting to user interactions, Real-Time is the key to open up a world of use cases that batch and scheduled processing cannot efficiently satisfy. In this talk, we are going to show some example use cases that Data Reply developed for some of its customers and how Real-Time Decision Engines had an impact on their businesses.
Hybrid Streaming Analytics for Apache Kafka Users | Firat Tekiner, Google
The data that organizations are required to analyze in order to make informed decisions is growing at an unprecedented rate. Companies have to capture the window of opportunity and become not just data driven, but event driven. In this talk, we will talk around addressing these issues and look into ways to bridge the on-premise kafka deployments with GCP stack for different use cases and personas. This will be followed by architecture examples on How do you deploy kafka and integrate with the rest of the GCP stack.
Transform Your Mainframe Data for the Cloud with Precisely and Apache Kafka
Your mainframe does hard work for your business, supporting essential computing transactions every day. However, mainframe data does not easily integrate with the cloud platforms driving data-driven, real-time, analytics-focused business processes. Integrating data from this critical technology often results in high costs and downtime. So, what can you do?
View this on-demand webinar to learn how Precisely Connect can help use the power of Apache Kafka to eliminate data silos and make cloud-based, event-driven data architectures a reality. Start your cloud transformation journey today, knowing you don’t need to leave essential transaction data behind!
During this webinar, you will learn more about:
· Where to begin your cloud transformation journey using mainframe data and Apache Kafka
· What you need to move mainframe data to the cloud while reducing costs, modernizing architectures, and using the staff you have today
· How Precisely Connect customers are using change data capture and Apache Kafka to deliver real-time insights to the cloud
Apache Kafka® Use Cases for Financial Services
Traditional systems were designed in an era that predates large-scale distributed systems. These systems often lack the ability to scale to meet the needs of the modern data-driven organisation. Adding to this is the accumulation of technologies and the explosion of data which can result in complex point-to-point integrations where data becomes siloed or separated across the enterprise.
The demand for fast results and decision making, have generated the need for real-time event streaming and processing of data adoption in financial institutions to be on the competitive edge. Apache Kafka and the Confluent Platform are designed to solve the problems associated with traditional systems and provide a modern, distributed architecture and Real-time Data streaming capability. In addition these technologies open up a range of use cases for Financial Services organisations, many of which will be explored in this talk. .
Recommended
Executive Briefing: What Is Fast Data And Why Is It Important
[About This Webinar]
Streaming data systems, so called Fast Data, promise accelerated access to information, leading to new innovations and competitive advantages. These systems, however, aren’t just faster versions of Big Data; they force architecture changes to meet new demands for reliability and dynamic scalability, more like microservices.
This means new challenges for your organization. Whereas a batch job might run for hours, a stream processing application might run for weeks or months. This raises the bar for making these systems resilient against traffic spikes, hardware and network failures, and so forth. The good news is that there is a strong history of facing these demands in the world of microservices.
In this webinar by Dr. Dean Wampler, VP of Fast Data Architecture at Lightbend, Inc., we will cut through the buzz around Fast Data and explore how to successfully exploit this new opportunity for innovation in how your organization leverages data. Specifically, Dean will review:
* The business justification for transitioning from batch-oriented big data to stream-oriented fast data
* The architectural and organizational changes that streaming systems require to meet their higher demands for reliability, resiliency, dynamic scalability, etc.
* How some of these requirements can be met by leveraging what your organization already knows about microservice architectures
What's The Role Of Machine Learning In Fast Data And Streaming Applications?
Designed for busy Architects and Managers, this Lightbend Webinar features Dr. Emre Velipasaoglu, Principal Data Scientist at Lightbend, who explains what ML is really all about, the ideal use cases for ML, and how getting it right can benefit your streaming and Fast Data application architectures.
[INFOGRAPHIC] Event-driven Business: How to Handle the Flow of Event Data
Get key research stats on why CIOs are turning to streaming data for a competitive advantage.
Improving Veteran benefit services through efficient data streaming | Robert ...
Having information on our Veterans and Veteran services is not an issue. The VA (VETERANS AFFAIRS) currently has petabytes of data on Veterans, the services they receive and the level of benefit to which those services align. The challenge we are aiming to solve through Confluent is the ability to publish eventing data in a manner consumable by various products - be they in–house applications or VA mobile apps - to rapidly notify a Veteran or service provider about key information concerning a claim or services received. Such information could include a change in status of a claim, requests for additional documentation to support a claim or appeal or what information should be shared when calling into the call center or what level of benefits are due when checking in at a hospital. This technology will enable the VA to provide accurate, real-time information on a claim, appeal or rating for our Veterans.
Data reply sneak peek: real time decision engines
Events happen constantly in every business: a purchase in an online shop, a credit limit is hit, the mobile internet plan has been exhausted, users interact with a website. Events rule the business world. So why would you react to them hours or days later? Real-Time Decision Engines enable a variety of use cases, driving new products, increasing user experience, reducing costs and risks by reacting instantly to business events.
From personalized instantaneous marketing campaigns to reacting to user interactions, Real-Time is the key to open up a world of use cases that batch and scheduled processing cannot efficiently satisfy. In this talk, we are going to show some example use cases that Data Reply developed for some of its customers and how Real-Time Decision Engines had an impact on their businesses.
Hybrid Streaming Analytics for Apache Kafka Users | Firat Tekiner, Google
The data that organizations are required to analyze in order to make informed decisions is growing at an unprecedented rate. Companies have to capture the window of opportunity and become not just data driven, but event driven. In this talk, we will talk around addressing these issues and look into ways to bridge the on-premise kafka deployments with GCP stack for different use cases and personas. This will be followed by architecture examples on How do you deploy kafka and integrate with the rest of the GCP stack.
Transform Your Mainframe Data for the Cloud with Precisely and Apache Kafka
Your mainframe does hard work for your business, supporting essential computing transactions every day. However, mainframe data does not easily integrate with the cloud platforms driving data-driven, real-time, analytics-focused business processes. Integrating data from this critical technology often results in high costs and downtime. So, what can you do?
View this on-demand webinar to learn how Precisely Connect can help use the power of Apache Kafka to eliminate data silos and make cloud-based, event-driven data architectures a reality. Start your cloud transformation journey today, knowing you don’t need to leave essential transaction data behind!
During this webinar, you will learn more about:
· Where to begin your cloud transformation journey using mainframe data and Apache Kafka
· What you need to move mainframe data to the cloud while reducing costs, modernizing architectures, and using the staff you have today
· How Precisely Connect customers are using change data capture and Apache Kafka to deliver real-time insights to the cloud
Apache Kafka® Use Cases for Financial Services
Traditional systems were designed in an era that predates large-scale distributed systems. These systems often lack the ability to scale to meet the needs of the modern data-driven organisation. Adding to this is the accumulation of technologies and the explosion of data which can result in complex point-to-point integrations where data becomes siloed or separated across the enterprise.
The demand for fast results and decision making, have generated the need for real-time event streaming and processing of data adoption in financial institutions to be on the competitive edge. Apache Kafka and the Confluent Platform are designed to solve the problems associated with traditional systems and provide a modern, distributed architecture and Real-time Data streaming capability. In addition these technologies open up a range of use cases for Financial Services organisations, many of which will be explored in this talk. .
What does an event mean? Manage the meaning of your data! | Andreas Wombacher...
Van Oord, a 150 year old family owned business, build windmill parks in the sea, lay cables on sea surface, dredging, as well as infrastructure (Dike, etc) operates world-wide, often facilitating self-owned specialized vessels. A well-known prestigious project is the creation of the palm island at the coast of Dubai.
Data Management in Van Oord is still in its infancy. The current operation is based on bilateral data exchange, without an Enterprise Service Bus or mayor data warehouse infrastructure. In 2020 Van Oord started a PoC with Confluent Kafka, executing a wide range of uses cases and requirements, followed by the formal program implementing a sustainable data platform.
Data owners are publishing an information product, i.e. a set of Kafka topics to communicate change (a la CDC) and topics for sharing state of a data source (Kafka tables). The information product owner is responsible for granting access, assuring data quality, data linage and governance. The set of all information products forms the enterprise data model.
This talk outlines why Van Oord requires data governance and enterprise architecture models integrated with Confluent Kafka, and demo how an open-source based data governance tool is integrated with Confluent Kafka to fulfil these requirements.
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
(Chris Maier + Steven Royster, West Monroe Partners) Kafka Summit SF 2018
The value of real-time data is growing as an increasing number of companies look to provide a comprehensive experience for their customers. Utilizing Kafka in key facets of your organization will yield greater customer satisfaction and promote a better understanding of user interactions. As data streaming is becoming more prevalent in a wide variety of industries, companies are seeking to modernize their tech stacks by employing the extensible, scalable infrastructure afforded by Kafka.
Over the past few months, we have successfully developed a containerized Kafka implementation at a major healthcare provider. In addition, we created producers to publish messages to the Kafka cluster and consumers to receive them on the other end. By capturing a plethora of data around customer activity, we created opportunities for the business to act upon real-time metrics in order to provide an improved customer experience.
In this talk, we will cover the user-related data sources we connected to Kafka, the reasons we chose them, and how the insights gained from each source can be leveraged in your business. You will walk out understanding how capturing a wide variety of customer activity data can create opportunities for the business to act on real-time metrics in order to provide an improved customer experience.
Confluent x imply: Build the last mile to value for data streaming applications
Build the last mile to value for data streaming applications
Streaming real time data with Vibe Data Stream
The process of streaming real-time data from a wide variety of machine data sources and entities can be very complex and unwieldy. Using an agent-based approach, Informatica has invented a new technique and open access product that makes this process much more user friendly and efficient, even when dealing with multiple environments such as Hadoop, Cassandra, Storm, Amazon Kinesis and Complex Event Processing.
Event-Streaming verstehen in unter 10 Min
Um die unternehmerische Geschwindigkeit zu erhöhen, die Wettbewerbsfähigkeit durch neue Produkte und Services zu steigern und schnell auf plötzlich ändernde Markteinflüsse reagieren zu können, müssen Daten und Ereignisströme in Echtzeit geteilt, verarbeitet und ausgewertet werden können. Apache Kafka hat sich hier als Industrie-Standard für Event-Streaming etabliert. Ob Connected Car, Industrie 4.0 oder Customer 360 – alle diese zukunftsorientierten Themen benötigen schnelle Kommunikation, effiziente Vernetzung und eine Verarbeitung von enormen Datenmengen in Echtzeit.
Webinar Data Mesh - Part 3
this is part 3 of the series on Data Mesh ... looking at the intersection of microservices architecture concepts, data integration / replication technologies and log-based stream integration techniques. This webinar was mostly a demonstration, but several slides used to setup the demo are included here as a PDF for viewers.
APAC Confluent Consumer Data Right the Lowdown and the Lessons
Watch the replay here: https://videos.confluent.io/watch/mnPk2Y1ibvhCsUhsdS1qrM?
Speaker Name: David Peterson
Bridge Your Kafka Streams to Azure Webinar
With a fully managed Apache Kafka(R) as-a-service on Microsoft Azure, businesses can focus on building applications and not managing clusters. Build a persistent bridge from on-premises data systems to the cloud with a hybrid Kafka service or stream across public clouds for multi-cloud data pipelines.
In this session for business and technical data leaders, you can learn about powering business applications with the managed Kafka service that streams data into Azure SQL Data Warehouse, Cosmos DB, Azure Data Lake Storage and Azure Blob Storage.
Financial Event Sourcing at Enterprise Scale
For years, Rabobank has been actively investing in becoming a real-time, event-driven bank. If you are familiar with banking processes, you will understand that this is not simple. Many banking processes are implemented as batch jobs on not-so-commodity hardware, meaning that any migration effort is immense.
*Find out how Rabobank redesigned Rabo Alerts while continuing to provide a robust and stable alert system for its existing user base
*Learn how the project team managed to achieve a balance between the need to decentralise activity while not losing control
*Understand how Rabobank re-invented a reliable service to meet modern customer expectations
Event Streaming Architecture for Industry 4.0 - Abdelkrim Hadjidj & Jan Kuni...
New use cases under the Industry 4.0 umbrella are playing a key role in improving factory operations, process optimization, cost reduction and quality improvement. We propose an event streaming architecture to streamline the information flow all the way from the factory to the main data center. Building such a streaming architecture enables a manufacturer to react faster to critical operational events. However, it presents two main challenges:
Data acquisition in real time: data should be collected regardless of its location or access challenges are. It is commonplace to ingest data from hundreds of heterogeneous data sources (ERP, MES, Sensors, maintenance systems, etc).
Event processing in real time: events collected from different parts of the organization should be combined into actionable insights in real time. This is extremely challenging in a context where events can be lost or delayed.
In this talk, we show how Apache NiFi and MiNiFi can be used to collect a wide range of datasources in real-time, connecting the industrial and information worlds. Then, we show how Apache Flink’s unique features enables us to make sense of this data. For instance, we will explain how Flink’s time management such Event Time mode, late arrival handling and watermark mechanism can be used to address the challenge of processing IoT data originating from geographically distributed plants. Finally, we demonstrate an end to end streaming architecture for Industry 4.0 based on the Cloudera DataFlow platform.
Modernizing your Application Architecture with Microservices
Organizations are quickly adopting microservice architectures to achieve better customer service and improve user experience while limiting downtime and data loss. However, transitioning from a monolithic architecture based on stateful databases to truly stateless microservices can be challenging and requires the right set of solutions.
In this webinar, learn from field experts as they discuss how to convert the data locked in traditional databases into event streams using HVR and Apache Kafka®. They will show you how to implement these solutions through a real-world demo use case of microservice adoption.
You will learn:
-How log-based change data capture (CDC) converts database tables into event streams
-How Kafka serves as the central nervous system for microservices
-How the transition to microservices can be realized without throwing away your legacy infrastructure
How to Quantify the Value of Kafka in Your Organization
(Lyndon Hedderly, Confluent) Kafka Summit SF 2018
We all know real-time data has a value. But how do you quantify that value in order to create a business case for becoming more data, or event driven?
The first half of this talk will explore the value of data across a variety of organizations, starting with the five most valuable companies in the world: Apple, Alphabet (Google), Microsoft, Amazon and Facebook (based on stock prices July 2017). We will go on to discuss other digital natives: Uber, Ebay, Netflix and LinkedIn, before exploring more traditional companies across retail, finance and automotive. Next, we’ll look at non-businesses such as governments and lobbyists. Whether organizations are using data to create new business products and services, improve user experiences, increase productivity, manage risk or influencing global power, we’ll see that fast and interconnected data, or “event streaming” is increasingly important.
After showing that data value can be quantified, the second half of this talk will explain the five steps to creating a business case.
Most businesses focus on:
-Making more money or conferring competitive advantage to make more money
-Increasing efficiency to save money and/or
-Mitigating risk to the business to protect money
-We’ll walk through examples of real business cases, discuss how business cases have evolved over the years and show the power of a sound business case. If you’re interested in big money and big business, as well as big data, this talk is for you.
Building a Secure, Tamper-Proof & Scalable Blockchain on Top of Apache Kafka ...
Apache Kafka is an open source event streaming platform. It is often used to complement or even replace existing middleware to integrate applications and build microservice architectures. Apache Kafka is already used in various projects in almost every bigger company today. Understood, battled-tested, highly scalable, reliable, real-time.
Blockchain is a different story. This technology is a lot in the news, especially related to cryptocurrencies like Bitcoin. But what is the added value for software architectures? Is blockchain just hype and adds complexity? Or will it be used by everybody in the future, like a web browser or mobile app today? And how is it related to an integration architecture and event streaming platform?
This session explores use cases for blockchains and discusses different alternatives such as Hyperledger, Ethereum and a Kafka-native tamper-proof blockchain implementation. Different architectures are discussed to understand when blockchain really adds value and how it can be combined with the Apache Kafka ecosystem to integrate blockchain with the rest of the enterprise architecture to build a highly scalable and reliable event streaming infrastructure.
Speakers:
Kai Waehner, Technology Evangelist, Confluent
Stephen Reed, CTO, Co-Founder, AiB
Intelligent Auto-scaling of Kafka Consumers with Workload Prediction | Ming S...
In a typical deployment of Kafka with many topics and partitions, scaling the Kafka consumer efficiently is one of the important tasks in maintaining overall smooth Kafka operations. The traditional Kubernetes Horizontal Pod Scaling (HPA) that uses basic CPU and/or memory metrics is not suitable for scaling Kafka consumers. The more appropriate workload metric for Kafka consumer is the number of messages in Kafka broker queue. More specifically, the message production rate of a specific topic would be the right workload metric for a Kafka consumer.
While using message production rate is a better way to decide the number of consumer replicas, this is still a reaction based auto-scaling. Using machine-learning based forecasting, it is possible for predict the upcoming increase or decrease of message production rate. With predicted workload, scaling the Kafka consumers could be achieved in a more timely manner, resulting with better performance KPI's.
Highly configurable and extensible data processing framework at PubMatic
PubMatic is a leading advertisement technology company that processes 500 billion transactions (50 terabytes of data) per day in real-time and batch processing pipeline on a 900-node cluster to power highly efficient machine learning algorithms, provide real time feedback to ad-server for optimization and provide in depth insights on customer inventory and audience.
At PubMatic, scaling with ever growing volume has always been the biggest challenge; we have been optimizing our technology stack for performance and costs. Another challenge is to support the demand for variety reports and analytics by customers and internal stakeholders. Writing custom jobs to provide analytics leads to repetitive efforts and redundancy of business logic in many different jobs.
To solve the above problems, we built a platform that allows creating configuration driven data processing pipeline with high re-usability of business functions. It is also extensible to utilize cutting-edge technologies in the ever-changing big data ecosystem. This platform enables our development teams to build a robust batch data processing pipeline to power analytics dashboards. It also empowers novice users to provide a configuration with fact and dimensions to generate ad-hoc reports in a single data processing job. Framework intelligently identifies and re-uses existing business functions based on user inputs. It also provides an abstraction layer that keeps core business logic un-affected by the any technology changes. This framework is currently powered by Spark, but it can be easily configured with other technologies.
Framework significantly improved time to develop data processing jobs from weeks to few days, it simplified unit testing and QA automation, as well as provided simpler interfaces to the customers and internal stakeholders to generate custom reports.
Speaker
Kunal Umrigar, Sr. Director Engineering Big Data & Analytics, PubMatic
Risk Management in Retail with Stream Processing (Daniel Jagielski, Virtuslab...
Every 2 seconds, another person becomes a victim of identity theft. The number of online account takeovers is constantly increasing. In this talk we'll show how stream processing was used to combat this for Tesco, one of Europe's largest retailers. The massive scale of e-commerce makes it an interesting target for malicious users. We implemented a risk-management platform built around Kafka and the Confluent Platform to detect and prevent attacks, including those that come through the website's authentication page. We'll present how this project evolved over 2 years to its current state in production, together with some of the challenges we encountered on the way. As the project has had a couple of phases, we will see and compare alternative designs, summarize their pros and cons, and refer them to well known techniques - like Event Sourcing. We'll discuss the architecture and integration with external systems, before moving onto a detailed examination of the stream processors implementation and key internals such as co-partitioning of data. We'll also cover the role of stack components that we used, including Kafka Connect and Schema Registry, as well as the deployment platform, Kubernetes. Over the course of the talk we will put special emphasis on highlighting key factors to take into consideration when designing data pipelines and stream processing platform.
Building an Enterprise Eventing Framework
Watch this talk here: https://www.confluent.io/online-talks/building-an-enterprise-eventing-framework-on-demand
Learn how Centene improved their ability to interact and engage with healthcare providers in real time with MongoDB and Confluent Platform.
Centene is fundamentally modernizing its legacy monolithic systems to support distributed, real-time event-driven healthcare information processing. A key part of their architecture is the development of a universal eventing framework designed to accommodate transformation into an event-driven architecture (EDA).
The business requirements within Centene's claims adjudication domain were solved leveraging the Kafka Stream DSL, Confluent Platform and MongoDB. Most importantly, Centene discusses how they plan on leveraging this framework to change their culture from batch processing to real-time stream processing.
Stream me to the Cloud (and back) with Confluent & MongoDB
In this online talk, we’ll explore how and why companies are leveraging Confluent and MongoDB to modernize their architecture and leverage the scalability of the cloud and the velocity of streaming. Based upon a sample retail business scenario, we will explain how changes in an on-premise database are streamed via the Confluent Cloud to MongoDB Atlas and back.
Confluent Cloud for Apache Kafka® | Google Cloud Next ’19
Google Cloud Next ’19
Speakers:
Gaetan Castelein, Confluent Product Marketing
Kir Titievsky, Google Product Management
Confluent Cloud for Apache Kafka® was a session conducted at Google Cloud Next ’19 on the topic of how Confluent and Google are partnering to give you a complete event-streaming platform that extends Kafka with essential capabilities for developers and enterprises. Confluent is available as a fully managed, first class service on GCP, or can be deployed on-premises on Google Cloud Services Platform. Developers can deploy Confluent Cloud™ in minutes right from the Google Cloud Console to start building event-driven applications. Enterprises can build hybrid cloud streaming solutions with a common platform that spans from on-premises to GCP, streaming data to GCP to leverage best-of-breed services such as BigQuery and TensorFlow. Review this presentation to learn about Confluent and GCP services, and see how you can get started in just minutes with no upfront commitment.
Apache Kafka as Event Streaming Platform for Microservice Architectures
This session introduces Apache Kafka, an event-driven open source streaming platform. Apache Kafka goes far beyond scalable, high volume messaging. In addition, you can leverage Kafka Connect for integration and the Kafka Streams API for building lightweight stream processing microservices in autonomous teams. The Confluent Platform adds further components such as a Schema Registry, REST Proxy, KSQL, Clients for different programming languages and Connectors for different technologies.
The session discusses how tech giants like LinkedIn, Ebay or Airbnb leverage Apache Kafka as event streaming platform to solve various different business problems and how to create a scalable, flexible microservice architecture. A live demo shows how you can easily process and analyze streams of events using Apache Kafka and KSQL.
More Related Content
What's hot
What does an event mean? Manage the meaning of your data! | Andreas Wombacher...
Van Oord, a 150 year old family owned business, build windmill parks in the sea, lay cables on sea surface, dredging, as well as infrastructure (Dike, etc) operates world-wide, often facilitating self-owned specialized vessels. A well-known prestigious project is the creation of the palm island at the coast of Dubai.
Data Management in Van Oord is still in its infancy. The current operation is based on bilateral data exchange, without an Enterprise Service Bus or mayor data warehouse infrastructure. In 2020 Van Oord started a PoC with Confluent Kafka, executing a wide range of uses cases and requirements, followed by the formal program implementing a sustainable data platform.
Data owners are publishing an information product, i.e. a set of Kafka topics to communicate change (a la CDC) and topics for sharing state of a data source (Kafka tables). The information product owner is responsible for granting access, assuring data quality, data linage and governance. The set of all information products forms the enterprise data model.
This talk outlines why Van Oord requires data governance and enterprise architecture models integrated with Confluent Kafka, and demo how an open-source based data governance tool is integrated with Confluent Kafka to fulfil these requirements.
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
(Chris Maier + Steven Royster, West Monroe Partners) Kafka Summit SF 2018
The value of real-time data is growing as an increasing number of companies look to provide a comprehensive experience for their customers. Utilizing Kafka in key facets of your organization will yield greater customer satisfaction and promote a better understanding of user interactions. As data streaming is becoming more prevalent in a wide variety of industries, companies are seeking to modernize their tech stacks by employing the extensible, scalable infrastructure afforded by Kafka.
Over the past few months, we have successfully developed a containerized Kafka implementation at a major healthcare provider. In addition, we created producers to publish messages to the Kafka cluster and consumers to receive them on the other end. By capturing a plethora of data around customer activity, we created opportunities for the business to act upon real-time metrics in order to provide an improved customer experience.
In this talk, we will cover the user-related data sources we connected to Kafka, the reasons we chose them, and how the insights gained from each source can be leveraged in your business. You will walk out understanding how capturing a wide variety of customer activity data can create opportunities for the business to act on real-time metrics in order to provide an improved customer experience.
Confluent x imply: Build the last mile to value for data streaming applications
Build the last mile to value for data streaming applications
Streaming real time data with Vibe Data Stream
The process of streaming real-time data from a wide variety of machine data sources and entities can be very complex and unwieldy. Using an agent-based approach, Informatica has invented a new technique and open access product that makes this process much more user friendly and efficient, even when dealing with multiple environments such as Hadoop, Cassandra, Storm, Amazon Kinesis and Complex Event Processing.
Event-Streaming verstehen in unter 10 Min
Um die unternehmerische Geschwindigkeit zu erhöhen, die Wettbewerbsfähigkeit durch neue Produkte und Services zu steigern und schnell auf plötzlich ändernde Markteinflüsse reagieren zu können, müssen Daten und Ereignisströme in Echtzeit geteilt, verarbeitet und ausgewertet werden können. Apache Kafka hat sich hier als Industrie-Standard für Event-Streaming etabliert. Ob Connected Car, Industrie 4.0 oder Customer 360 – alle diese zukunftsorientierten Themen benötigen schnelle Kommunikation, effiziente Vernetzung und eine Verarbeitung von enormen Datenmengen in Echtzeit.
Webinar Data Mesh - Part 3
this is part 3 of the series on Data Mesh ... looking at the intersection of microservices architecture concepts, data integration / replication technologies and log-based stream integration techniques. This webinar was mostly a demonstration, but several slides used to setup the demo are included here as a PDF for viewers.
APAC Confluent Consumer Data Right the Lowdown and the Lessons
Watch the replay here: https://videos.confluent.io/watch/mnPk2Y1ibvhCsUhsdS1qrM?
Speaker Name: David Peterson
Bridge Your Kafka Streams to Azure Webinar
With a fully managed Apache Kafka(R) as-a-service on Microsoft Azure, businesses can focus on building applications and not managing clusters. Build a persistent bridge from on-premises data systems to the cloud with a hybrid Kafka service or stream across public clouds for multi-cloud data pipelines.
In this session for business and technical data leaders, you can learn about powering business applications with the managed Kafka service that streams data into Azure SQL Data Warehouse, Cosmos DB, Azure Data Lake Storage and Azure Blob Storage.
Financial Event Sourcing at Enterprise Scale
For years, Rabobank has been actively investing in becoming a real-time, event-driven bank. If you are familiar with banking processes, you will understand that this is not simple. Many banking processes are implemented as batch jobs on not-so-commodity hardware, meaning that any migration effort is immense.
*Find out how Rabobank redesigned Rabo Alerts while continuing to provide a robust and stable alert system for its existing user base
*Learn how the project team managed to achieve a balance between the need to decentralise activity while not losing control
*Understand how Rabobank re-invented a reliable service to meet modern customer expectations
Event Streaming Architecture for Industry 4.0 - Abdelkrim Hadjidj & Jan Kuni...
New use cases under the Industry 4.0 umbrella are playing a key role in improving factory operations, process optimization, cost reduction and quality improvement. We propose an event streaming architecture to streamline the information flow all the way from the factory to the main data center. Building such a streaming architecture enables a manufacturer to react faster to critical operational events. However, it presents two main challenges:
Data acquisition in real time: data should be collected regardless of its location or access challenges are. It is commonplace to ingest data from hundreds of heterogeneous data sources (ERP, MES, Sensors, maintenance systems, etc).
Event processing in real time: events collected from different parts of the organization should be combined into actionable insights in real time. This is extremely challenging in a context where events can be lost or delayed.
In this talk, we show how Apache NiFi and MiNiFi can be used to collect a wide range of datasources in real-time, connecting the industrial and information worlds. Then, we show how Apache Flink’s unique features enables us to make sense of this data. For instance, we will explain how Flink’s time management such Event Time mode, late arrival handling and watermark mechanism can be used to address the challenge of processing IoT data originating from geographically distributed plants. Finally, we demonstrate an end to end streaming architecture for Industry 4.0 based on the Cloudera DataFlow platform.
Modernizing your Application Architecture with Microservices
Organizations are quickly adopting microservice architectures to achieve better customer service and improve user experience while limiting downtime and data loss. However, transitioning from a monolithic architecture based on stateful databases to truly stateless microservices can be challenging and requires the right set of solutions.
In this webinar, learn from field experts as they discuss how to convert the data locked in traditional databases into event streams using HVR and Apache Kafka®. They will show you how to implement these solutions through a real-world demo use case of microservice adoption.
You will learn:
-How log-based change data capture (CDC) converts database tables into event streams
-How Kafka serves as the central nervous system for microservices
-How the transition to microservices can be realized without throwing away your legacy infrastructure
How to Quantify the Value of Kafka in Your Organization
(Lyndon Hedderly, Confluent) Kafka Summit SF 2018
We all know real-time data has a value. But how do you quantify that value in order to create a business case for becoming more data, or event driven?
The first half of this talk will explore the value of data across a variety of organizations, starting with the five most valuable companies in the world: Apple, Alphabet (Google), Microsoft, Amazon and Facebook (based on stock prices July 2017). We will go on to discuss other digital natives: Uber, Ebay, Netflix and LinkedIn, before exploring more traditional companies across retail, finance and automotive. Next, we’ll look at non-businesses such as governments and lobbyists. Whether organizations are using data to create new business products and services, improve user experiences, increase productivity, manage risk or influencing global power, we’ll see that fast and interconnected data, or “event streaming” is increasingly important.
After showing that data value can be quantified, the second half of this talk will explain the five steps to creating a business case.
Most businesses focus on:
-Making more money or conferring competitive advantage to make more money
-Increasing efficiency to save money and/or
-Mitigating risk to the business to protect money
-We’ll walk through examples of real business cases, discuss how business cases have evolved over the years and show the power of a sound business case. If you’re interested in big money and big business, as well as big data, this talk is for you.
Building a Secure, Tamper-Proof & Scalable Blockchain on Top of Apache Kafka ...
Apache Kafka is an open source event streaming platform. It is often used to complement or even replace existing middleware to integrate applications and build microservice architectures. Apache Kafka is already used in various projects in almost every bigger company today. Understood, battled-tested, highly scalable, reliable, real-time.
Blockchain is a different story. This technology is a lot in the news, especially related to cryptocurrencies like Bitcoin. But what is the added value for software architectures? Is blockchain just hype and adds complexity? Or will it be used by everybody in the future, like a web browser or mobile app today? And how is it related to an integration architecture and event streaming platform?
This session explores use cases for blockchains and discusses different alternatives such as Hyperledger, Ethereum and a Kafka-native tamper-proof blockchain implementation. Different architectures are discussed to understand when blockchain really adds value and how it can be combined with the Apache Kafka ecosystem to integrate blockchain with the rest of the enterprise architecture to build a highly scalable and reliable event streaming infrastructure.
Speakers:
Kai Waehner, Technology Evangelist, Confluent
Stephen Reed, CTO, Co-Founder, AiB
Intelligent Auto-scaling of Kafka Consumers with Workload Prediction | Ming S...
In a typical deployment of Kafka with many topics and partitions, scaling the Kafka consumer efficiently is one of the important tasks in maintaining overall smooth Kafka operations. The traditional Kubernetes Horizontal Pod Scaling (HPA) that uses basic CPU and/or memory metrics is not suitable for scaling Kafka consumers. The more appropriate workload metric for Kafka consumer is the number of messages in Kafka broker queue. More specifically, the message production rate of a specific topic would be the right workload metric for a Kafka consumer.
While using message production rate is a better way to decide the number of consumer replicas, this is still a reaction based auto-scaling. Using machine-learning based forecasting, it is possible for predict the upcoming increase or decrease of message production rate. With predicted workload, scaling the Kafka consumers could be achieved in a more timely manner, resulting with better performance KPI's.
Highly configurable and extensible data processing framework at PubMatic
PubMatic is a leading advertisement technology company that processes 500 billion transactions (50 terabytes of data) per day in real-time and batch processing pipeline on a 900-node cluster to power highly efficient machine learning algorithms, provide real time feedback to ad-server for optimization and provide in depth insights on customer inventory and audience.
At PubMatic, scaling with ever growing volume has always been the biggest challenge; we have been optimizing our technology stack for performance and costs. Another challenge is to support the demand for variety reports and analytics by customers and internal stakeholders. Writing custom jobs to provide analytics leads to repetitive efforts and redundancy of business logic in many different jobs.
To solve the above problems, we built a platform that allows creating configuration driven data processing pipeline with high re-usability of business functions. It is also extensible to utilize cutting-edge technologies in the ever-changing big data ecosystem. This platform enables our development teams to build a robust batch data processing pipeline to power analytics dashboards. It also empowers novice users to provide a configuration with fact and dimensions to generate ad-hoc reports in a single data processing job. Framework intelligently identifies and re-uses existing business functions based on user inputs. It also provides an abstraction layer that keeps core business logic un-affected by the any technology changes. This framework is currently powered by Spark, but it can be easily configured with other technologies.
Framework significantly improved time to develop data processing jobs from weeks to few days, it simplified unit testing and QA automation, as well as provided simpler interfaces to the customers and internal stakeholders to generate custom reports.
Speaker
Kunal Umrigar, Sr. Director Engineering Big Data & Analytics, PubMatic
Risk Management in Retail with Stream Processing (Daniel Jagielski, Virtuslab...
Every 2 seconds, another person becomes a victim of identity theft. The number of online account takeovers is constantly increasing. In this talk we'll show how stream processing was used to combat this for Tesco, one of Europe's largest retailers. The massive scale of e-commerce makes it an interesting target for malicious users. We implemented a risk-management platform built around Kafka and the Confluent Platform to detect and prevent attacks, including those that come through the website's authentication page. We'll present how this project evolved over 2 years to its current state in production, together with some of the challenges we encountered on the way. As the project has had a couple of phases, we will see and compare alternative designs, summarize their pros and cons, and refer them to well known techniques - like Event Sourcing. We'll discuss the architecture and integration with external systems, before moving onto a detailed examination of the stream processors implementation and key internals such as co-partitioning of data. We'll also cover the role of stack components that we used, including Kafka Connect and Schema Registry, as well as the deployment platform, Kubernetes. Over the course of the talk we will put special emphasis on highlighting key factors to take into consideration when designing data pipelines and stream processing platform.
Building an Enterprise Eventing Framework
Watch this talk here: https://www.confluent.io/online-talks/building-an-enterprise-eventing-framework-on-demand
Learn how Centene improved their ability to interact and engage with healthcare providers in real time with MongoDB and Confluent Platform.
Centene is fundamentally modernizing its legacy monolithic systems to support distributed, real-time event-driven healthcare information processing. A key part of their architecture is the development of a universal eventing framework designed to accommodate transformation into an event-driven architecture (EDA).
The business requirements within Centene's claims adjudication domain were solved leveraging the Kafka Stream DSL, Confluent Platform and MongoDB. Most importantly, Centene discusses how they plan on leveraging this framework to change their culture from batch processing to real-time stream processing.
Stream me to the Cloud (and back) with Confluent & MongoDB
In this online talk, we’ll explore how and why companies are leveraging Confluent and MongoDB to modernize their architecture and leverage the scalability of the cloud and the velocity of streaming. Based upon a sample retail business scenario, we will explain how changes in an on-premise database are streamed via the Confluent Cloud to MongoDB Atlas and back.
Confluent Cloud for Apache Kafka® | Google Cloud Next ’19
Google Cloud Next ’19
Speakers:
Gaetan Castelein, Confluent Product Marketing
Kir Titievsky, Google Product Management
Confluent Cloud for Apache Kafka® was a session conducted at Google Cloud Next ’19 on the topic of how Confluent and Google are partnering to give you a complete event-streaming platform that extends Kafka with essential capabilities for developers and enterprises. Confluent is available as a fully managed, first class service on GCP, or can be deployed on-premises on Google Cloud Services Platform. Developers can deploy Confluent Cloud™ in minutes right from the Google Cloud Console to start building event-driven applications. Enterprises can build hybrid cloud streaming solutions with a common platform that spans from on-premises to GCP, streaming data to GCP to leverage best-of-breed services such as BigQuery and TensorFlow. Review this presentation to learn about Confluent and GCP services, and see how you can get started in just minutes with no upfront commitment.
What's hot (20)
What does an event mean? Manage the meaning of your data! | Andreas Wombacher...
What does an event mean? Manage the meaning of your data! | Andreas Wombacher...
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
Confluent x imply: Build the last mile to value for data streaming applications
Confluent x imply: Build the last mile to value for data streaming applications
APAC Confluent Consumer Data Right the Lowdown and the Lessons
APAC Confluent Consumer Data Right the Lowdown and the Lessons
Event Streaming Architecture for Industry 4.0 - Abdelkrim Hadjidj & Jan Kuni...
Event Streaming Architecture for Industry 4.0 - Abdelkrim Hadjidj & Jan Kuni...
Modernizing your Application Architecture with Microservices
Modernizing your Application Architecture with Microservices
How to Quantify the Value of Kafka in Your Organization
How to Quantify the Value of Kafka in Your Organization
Building a Secure, Tamper-Proof & Scalable Blockchain on Top of Apache Kafka ...
Building a Secure, Tamper-Proof & Scalable Blockchain on Top of Apache Kafka ...
Intelligent Auto-scaling of Kafka Consumers with Workload Prediction | Ming S...
Intelligent Auto-scaling of Kafka Consumers with Workload Prediction | Ming S...
Highly configurable and extensible data processing framework at PubMatic
Highly configurable and extensible data processing framework at PubMatic
Risk Management in Retail with Stream Processing (Daniel Jagielski, Virtuslab...
Risk Management in Retail with Stream Processing (Daniel Jagielski, Virtuslab...
Stream me to the Cloud (and back) with Confluent & MongoDB
Stream me to the Cloud (and back) with Confluent & MongoDB
Confluent Cloud for Apache Kafka® | Google Cloud Next ’19
Confluent Cloud for Apache Kafka® | Google Cloud Next ’19
Similar to Transforming The Customer Experience With Real-Time Insights
Apache Kafka as Event Streaming Platform for Microservice Architectures
This session introduces Apache Kafka, an event-driven open source streaming platform. Apache Kafka goes far beyond scalable, high volume messaging. In addition, you can leverage Kafka Connect for integration and the Kafka Streams API for building lightweight stream processing microservices in autonomous teams. The Confluent Platform adds further components such as a Schema Registry, REST Proxy, KSQL, Clients for different programming languages and Connectors for different technologies.
The session discusses how tech giants like LinkedIn, Ebay or Airbnb leverage Apache Kafka as event streaming platform to solve various different business problems and how to create a scalable, flexible microservice architecture. A live demo shows how you can easily process and analyze streams of events using Apache Kafka and KSQL.
Replicate Salesforce Data in Real Time with Change Data Capture
Migrate your batch processing, scheduled ETL, and nightly workloads to event-driven, real-time integrations using Change Data Capture. CDC means data change events are published to an event stream, allowing businesses to have up-to-date information across systems and applications. Join us to learn how to configure Change Data Capture and subscribe to the stream of change events, streamlining your architectures and processes.
Architecting Petabyte Scale AI Applications
Presented at the AI Summit SF by Ganesh Harinath, VP of Engineering, Big Data and Artificial Intelligence, Oath.
Azure IPaaS: #IntegrationEvolved (Glenn Colpaert @ Codit's BizTalk 2016 Launch)
Once upon a time… integration was all about ESB’s, EAI and B2B. Today, many companies wish to integrate beyond firewalls, and typically with SaaS. Hence the uprise of API based integration, using lightweight protocols. The evolution is a fact; so now what is the current state of the Azure Integration Platform?
Glenn dives into its architecture and explains Logic Apps and the Enterprise Integration Pack. Learn to create basic IFTTT (If This Then That) scenarios, or why not: think bigger and create enterprise-level, hybrid integration scenarios using Logic Apps and on premises LOB apps. 'How does it work', 'How is it Made' and 'How does it all fit together’? Just a couple of questions you will find the answer to.
Georgia Azure Event - Scalable cloud games using Microsoft Azure
How to build scalable cloud games using Microsoft Azure - with cases and architecture samples.
Simplified Machine Learning Architecture with an Event Streaming Platform (Ap...
Machine Learning is separated into model training and model inference. ML frameworks typically load historical data from a data store like HDFS or S3 to train models. This talk shows how you can completely avoid such a data store by ingesting streaming data directly via Apache Kafka from any source system into TensorFlow for model training and model inference using the capabilities of “TensorFlow I/O” add-on.
The talk compares this modern streaming architecture to traditional batch and big data alternatives and explains benefits like the simplified architecture, the ability of reprocessing events in the same order for training different models, and the possibility to build a scalable, mission-critical, real time ML architecture with muss less headaches and problems.
Key takeaways for the audience
• Scalable open source Machine Learning infrastructure
• Streaming ingestion into TensorFlow without the need for another data store like HDFS or S3 (leveraging TensorFlow I/O and its Kafka plugin)
• Stream Processing using analytic models in mission-critical deployments to act in Real Time
• Learn how Apache Kafka open source ecosystem including Kafka Connect, Kafka Streams and KSQL help to build, deploy, score and monitor analytic models
• Comparison and trade-offs between this modern streaming approach and traditional batch model training infrastructures
iPlanet presentation
Great presentation but the Netscape/Sun/iPlanet vision in 1999. What have we done since then
APIsecure 2023 - API orchestration: to build resilient applications, Cherish ...
APIsecure 2023 - The world's first and only API security conference
March 14 & 15, 2023
API orchestration: to build resilient applications
Cherish Santoshi, Sr. Developer Relations Engineer at Orkes
------
Check out our conferences at https://www.apidays.global/
Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8
Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io
Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/
Architect day 20181128 - Afternoon Session
This is the content that Salesforce shared on Architect Day Stockholm.
Motadata brochure
Mindarray Systems has been an innovator in Network Monitoring and Data Analytics Technologies. Our first product, Minder, was the only IT monitoring platform that offered a truly unified view and correlation across all layers of IT infrastructure from a single product.
Motadata today, is strongly positioned to enable Telco’s, Enterprise and Government sectors to streamline their IT operations with next-gen product suite which is pre-integrated yet modular, consisting of
1) Infrastructure Intelligence Platform (IIP) for Network Management
2) Data Analytics Platform (DAP) for Log Management & Flow Analytics and
3) ITSM Platform for IT Service Automation.
Motadata is a unified IT Infrastructure Monitoring, Log & Flow Management, and IT Service Management Platform, offering operational insights into your IT infrastructure and its performance and is designed to identify & resolve complex problems faster that ensures 100% uptime of all business-critical components. Motadata enables you to make more informed business decisions by offering complete visibility into the health and key performance indicators (KPIs) of IT services. It helps in reducing CAPEX, offers Agility to resolve issues faster, is compatible in a hybrid ecosystem, and offers ease of integration with existing and future platforms.
In summary, with Motadata, Mindarray Systems offers the perfect solution needed to confidently handle the challenges of today’s increasingly complex business operations and IT infrastructure management.
The 4th Generation Kingland platform
15 slide presentation displaying the use cases, features and benefits of the 4th generation Kingland Platform. The platform delivers enterprise data management solutions for some of the world's largest organizations. Powered by an artificial intelligence suite, the platform helps organizations avoid costs, accelerate projects, and improve how you use data to make business decisions.
Stream analytics
StreamAnalytix, a product of Impetus Technologies, enables enterprises to analyse and respond to events in real-time at Big Data scale.
Similar to Transforming The Customer Experience With Real-Time Insights (20)
Apache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice Architectures
Replicate Salesforce Data in Real Time with Change Data Capture
Replicate Salesforce Data in Real Time with Change Data Capture
Azure IPaaS: #IntegrationEvolved (Glenn Colpaert @ Codit's BizTalk 2016 Launch)
Azure IPaaS: #IntegrationEvolved (Glenn Colpaert @ Codit's BizTalk 2016 Launch)
Georgia Azure Event - Scalable cloud games using Microsoft Azure
Georgia Azure Event - Scalable cloud games using Microsoft Azure
Simplified Machine Learning Architecture with an Event Streaming Platform (Ap...
Simplified Machine Learning Architecture with an Event Streaming Platform (Ap...
Streaming Data and Stream Processing with Apache Kafka
Streaming Data and Stream Processing with Apache Kafka
APIsecure 2023 - API orchestration: to build resilient applications, Cherish ...
APIsecure 2023 - API orchestration: to build resilient applications, Cherish ...
More from Lightbend
IoT 'Megaservices' - High Throughput Microservices with Akka
**********
Watch this presentation on-demand!
https://info.lightbend.com/iot-megaservices-high-throughput-microservices-with-akka-register.html
**********
In this interactive presentation by Hugh McKee, Developer Advocate at Lightbend, we’ll share our experiences helping our clients create a system architecture that can support high throughput microservices (aka "Megaservices"). We’ll do that using IoT demo applications designed to push cloud service providers like Amazon and Google to their limits. Using sample code that you can later run on your own machine, we’ll look at:
* Modeling real-life digital twins for hundreds of thousands of IoT devices in the field, looking into how these megaservices are implemented in Akka.
* Visualizing Akka Actors–which represent IoT digital twins–in a “crop circle” formation that represents a complete distributed Reactive application, and watching at messages are processed across Akka Cluster nodes using cluster sharding.
* Some code behind the whole set up, which is built using OSS like Akka, Java, JavaScript, and Kubernetes.
Follow us on social:
TW: https://twitter.com/lightbend
LI: https://www.linkedin.com/company/lightbend-inc-/
FB: https://www.facebook.com/lightbendOfficial/
For more about Lightbend:
Blog: https://www.lightbend.com/blog
Newsletter: https://www.lightbend.com/newsletter
How Akka Cluster Works: Actors Living in a Cluster
Hugh McKee, Developer Advocate at Lightbend, demonstrates how Akka Actors work inside of a cluster, including the code and in-browser visualizations you need to grok it.
See the full content with videos here: https://www.lightbend.com/blog/how-akka-cluster-works-actors-living-in-a-cluster
The Reactive Principles: Eight Tenets For Building Cloud Native Applications
In this presentation by Jonas Bonér, creator of Akka and founder/CTO of Lightbend, we review a set of eight Reactive Principles that enable the design and implementation of Cloud Native applications–applications that are highly concurrent, distributed, performant, scalable, and resilient, while at the same time conserving resources when deploying, operating, and maintaining them.
Putting the 'I' in IoT - Building Digital Twins with Akka Microservices
In this webinar with Hugh McKee, Developer Advocate for Akka Platform, we’ll look at “What on Earth”, a demo exploring how Akka Microservices serves as an ideal solution for high-scale digital twinning for IoT.
For the full presentation, including video, visit: https://www.lightbend.com/blog/iot-building-digital-twins-with-akka-microservices
Akka at Enterprise Scale: Performance Tuning Distributed Applications
Organizations like Starbucks, HPE, and PayPal (see our customers) have selected the Akka toolkit for their enterprise scale distributed applications; and when it comes to squeezing out the best possible performance, the secret is using two particular modules in tandem: Akka Cluster and Akka Streams.
In this webinar by Nolan Grace, Senior Solution Architect at Lightbend, we look at these two Akka modules and discuss the features that will push your application architecture to the next tier of performance.
For the full blog post, including the video, visit: https://www.lightbend.com/blog/akka-at-enterprise-scale-performance-tuning-distributed-applications
Digital Transformation with Kubernetes, Containers, and Microservices
See the full presentation here: https://www.lightbend.com/blog/digital-transformation-kubernetes-containers-microservices
In this talk by David Ogren, Principal Enterprise Architect at Lightbend, we draw from experiences helping our clients successfully create, migrate to, and manage cloud-native system architectures.
Detecting Real-Time Financial Fraud with Cloudflow on Kubernetes
Deploying a robust streaming data pipeline can be a daunting task when your company’s financial information is at risk. For starters, how do you ensure proper provisioning of resources? How do you preserve end-to-end application and data consistency? How do you make all of this work in the cloud with Kubernetes and avoid YAML hell? Answer: Cloudflow, a new open-source toolkit for simplifying the development, deployment, and operation of streaming data pipelines.
Cloudstate - Towards Stateful Serverless
In this webinar by Jonas Bonér, creator of Akka and CTO/Co-Founder of Lightbend, we take a look at Cloudstate, an OSS tool built on Akka, gRPC, Knative, GraalVM, and Kubernetes. Cloudstate lets you model, manage, and scale stateful services while preserving responsiveness by designing for resilience and elasticity.
Digital Transformation from Monoliths to Microservices to Serverless and Beyond
Join this highly-visual presentation by Hugh McKee, Developer Advocate at Lightbend, to learn more about the ramifications and opportunities along the evolution from monolithic systems, to microservices architectures, to serverless (FaaS).
See the video presentation on the Lightbend blog at: https://www.lightbend.com/blog/digital-transformation-from-monoliths-to-microservices-to-serverless-and-beyond
Akka Anti-Patterns, Goodbye: Six Features of Akka 2.6
In this special guest webinar with Akka expert and Reactive System Consultant, Manuel Bernhardt, we review Akka 2.6 release highlights and a selection of 6 former anti-patterns that have now been rendered impossible by design.
Lessons From HPE: From Batch To Streaming For 20 Billion Sensors With Lightbe...
In this guest webinar with Chris McDermott, Lead Data Engineer at HPE, learn how HPE InfoSight–powered by Lightbend Platform–has emerged as the go-to solution for providing real-time metrics and predictive analytics across various network, server, storage, and data center technologies.
How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...
The Lightbend project family has grown: Introducing Cloudflow, the fastest way to build streaming data pipelines with Akka Streams, Flink, and Spark.
Microservices, Kubernetes, and Application Modernization Done Right
In this talk by David Ogren, Enterprise Architect at Lightbend, we draw from experiences helping our clients successfully create, migrate to, and manage cloud-native system architectures. We look at some of the common pitfalls and anti-patterns of modernization efforts, and some of the best practices for taking an incremental approach to transforming legacy systems.
See the full post with video on the Lightbend blog: https://www.lightbend.com/blog/microservices-kubernetes-application-modernization
Full Stack Reactive In Practice
In this guest webinar by Kevin Webber, we cover the entire architecture of a Reactive system, from a responsive UI implemented with Vue.js, to a fully event sourced collection of microservices implemented with Java, Lagom, Cassandra, and Kafka.
For the full recording, visit: https://www.lightbend.com/blog/full-stack-reactive-in-practice-webinar
Akka and Kubernetes: A Symbiotic Love Story
In this webinar by Hugh McKee, Developer Advocate at Lightbend, we take a look at how Akka and Kubernetes enjoy a symbiotic relationship, using live “crop circle” visuals to help. See the full video, slides, and additional resources here:
https://www.lightbend.com/blog/akka-and-kubernetes-a-symbiotic-love-story
Scala 3 Is Coming: Martin Odersky Shares What To Know
Join Dr. Martin Odersky, the creator of Scala and co-founder of Lightbend, on a tour of what is in store and highlight some of his favorite features of Scala 3!
Migrating From Java EE To Cloud-Native Reactive Systems
A lot of businesses that never before considered themselves as “technology companies” are now faced with digital modernization imperatives that force them to rethink their application and infrastructure architecture. On the path to becoming a digital, on-demand provider, development speed is the ultimate competitive advantage.
This presents challenges to many organizations that have huge investments in legacy Java EE infrastructure, where technical debt and monolithic system architectures require modernization in order to confront various business risks. Usually, changes need to be made within existing frameworks to keep pace with new web-scale organizations.
If your legacy monolith is no longer serving the expanding needs of your business, then join Markus Eisele, Director of Developer Advocacy at Lightbend, to learn what you can do to migrate from Java EE to cloud-native, Reactive systems—as defined by the Reactive Manifesto.
Running Kafka On Kubernetes With Strimzi For Real-Time Streaming Applications
In this talk by Sean Glover, Principal Engineer at Lightbend, we will review how the Strimzi Kafka Operator, a supported technology in Lightbend Platform, makes many operational tasks in Kafka easy, such as the initial deployment and updates of a Kafka and ZooKeeper cluster.
See the blog post containing the YouTube video here: https://www.lightbend.com/blog/running-kafka-on-kubernetes-with-strimzi-for-real-time-streaming-applications
Designing Events-First Microservices For A Cloud Native World
In this talk by Jonas Bonér, Lightbend CTO/Co-Founder and creator of Akka, we will explore the nature of events, what it means to be event-driven, and how we can unleash the power of events and commands by applying an events first, domain-driven design to microservices-based architectures.
For more information, head over to lightbend.com/blog!
Scala Security: Eliminate 200+ Code-Level Threats With Fortify SCA For Scala
Join Jeremy Daggett, Solutions Architect at Lightbend, to see how Fortify SCA for Scala works differently from existing Static Code Analysis tools to help you uncover security issues early in the SDLC of your mission-critical applications.
More from Lightbend (20)
IoT 'Megaservices' - High Throughput Microservices with Akka
IoT 'Megaservices' - High Throughput Microservices with Akka
How Akka Cluster Works: Actors Living in a Cluster
How Akka Cluster Works: Actors Living in a Cluster
The Reactive Principles: Eight Tenets For Building Cloud Native Applications
The Reactive Principles: Eight Tenets For Building Cloud Native Applications
Putting the 'I' in IoT - Building Digital Twins with Akka Microservices
Putting the 'I' in IoT - Building Digital Twins with Akka Microservices
Akka at Enterprise Scale: Performance Tuning Distributed Applications
Akka at Enterprise Scale: Performance Tuning Distributed Applications
Digital Transformation with Kubernetes, Containers, and Microservices
Digital Transformation with Kubernetes, Containers, and Microservices
Detecting Real-Time Financial Fraud with Cloudflow on Kubernetes
Detecting Real-Time Financial Fraud with Cloudflow on Kubernetes
Digital Transformation from Monoliths to Microservices to Serverless and Beyond
Digital Transformation from Monoliths to Microservices to Serverless and Beyond
Akka Anti-Patterns, Goodbye: Six Features of Akka 2.6
Akka Anti-Patterns, Goodbye: Six Features of Akka 2.6
Lessons From HPE: From Batch To Streaming For 20 Billion Sensors With Lightbe...
Lessons From HPE: From Batch To Streaming For 20 Billion Sensors With Lightbe...
How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...
How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...
Microservices, Kubernetes, and Application Modernization Done Right
Microservices, Kubernetes, and Application Modernization Done Right
Scala 3 Is Coming: Martin Odersky Shares What To Know
Scala 3 Is Coming: Martin Odersky Shares What To Know
Migrating From Java EE To Cloud-Native Reactive Systems
Migrating From Java EE To Cloud-Native Reactive Systems
Running Kafka On Kubernetes With Strimzi For Real-Time Streaming Applications
Running Kafka On Kubernetes With Strimzi For Real-Time Streaming Applications
Designing Events-First Microservices For A Cloud Native World
Designing Events-First Microservices For A Cloud Native World
Scala Security: Eliminate 200+ Code-Level Threats With Fortify SCA For Scala
Scala Security: Eliminate 200+ Code-Level Threats With Fortify SCA For Scala
Recently uploaded
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Mind map of terminologies used in context of Generative AI
Mind map of common terms used in context of Generative AI.
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Epistemic Interaction - tuning interfaces to provide information for AI support
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
UiPath Test Automation using UiPath Test Suite series, part 5
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Building RAG with self-deployed Milvus vector database and Snowpark Container...
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Recently uploaded (20)
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Transforming The Customer Experience With Real-Time Insights
- 2. Situation Vision Strategy Struggles Roadmap
- 4. Vision 1 2 3 4 5 Scale Development Real-Time Broad Insights Diverse Use Cases Agile Infrastructure
- 7. Streaming and Fast Data IoT Core Domain API Query DB Data Warehouse or Data Mart Device Shadow Service Device Shadows Device Streaming Load Streaming Load Stateless API Microservice Data Science KafkaSubscribe to Event Streams from other Domains Domain-Produced Event Stream Config Rule Checking Device Device Shadow Commands / Queries Status, Metrics, Events, Logs Streaming Analytics / ML
- 8. Kafka Device Auth Federated ID Entitlement CRM Cases IoT Core Domains Installed Base Notification Rule Registry API Gateway Device Shadow Wellness Tagging Microservices
- 10. Container Orchestration Registry Networking Storage Security Telemetry = Orchestration Containers
- 13. Pro-Tips 1. Rationalize your buzzwords 2. Domain-Driven Design 3. Stream all the things… 4. But not all at once 5. POC the hard parts 6. Get help early on 7. Keep going 8. Fail fast
- 14. Questions