Akka Streams is an implementation of Reactive Streams, which is a standard for asynchronous stream processing with non-blocking backpressure on the JVM. In this talk we'll cover the rationale behind Reactive Streams, and explore the different building blocks available in Akka Streams. I'll use Scala for all coding examples, but Akka Streams also provides a full-fledged Java8 API.After this session you will be all set and ready to reap the benefits of using Akka Streams!
Akka Streams is an implementation of Reactive Streams, which is a standard for asynchronous stream processing with non-blocking backpressure on the JVM. In this talk we'll cover the rationale behind Reactive Streams, and explore the different building blocks available in Akka Streams. I'll use Scala for all coding examples, but Akka Streams also provides a full-fledged Java8 API.After this session you will be all set and ready to reap the benefits of using Akka Streams!
Akka Streams are an implementation of the Reactive Streams specification (http://reactive-streams.org/), a joint effort that aims at standardizing the exchange of streams of data across asynchronous boundaries in a fully non-blocking way while providing flow control and mediating back pressure. In this presentation we go into the details of what this new abstraction can be used for and what the guiding principles are behind its development. We then focus on one prominent use-case which is the upcoming Akka HTTP module: a fully stream-enabled, reactive HTTP server and client implementation.
Akka Streams is a toolkit for processing of streams. It is an implementation of Reactive Streams Specification. Its purpose is to “formulate stream processing setups such that we can then execute them efficiently and with bounded resource usage.”
Journey into Reactive Streams and Akka StreamsKevin Webber
Are streams just collections? What's the difference between Java 8 streams and Reactive Streams? How do I implement Reactive Streams with Akka? Pub/sub, dynamic push/pull, non-blocking, non-dropping; these are some of the other concepts covered. We'll also discuss how to leverage streams in a real-world application.
Akka Streams (0.7) talk for the Tokyo Scala User Group, hosted by Dwango.
Akka streams are an reactive streams implementation which allows for asynchronous back-pressured processing of data in complext pipelines. This talk aims to highlight the details about how reactive streams work as well as some of the ideas behind akka streams.
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Ka...Reactivesummit
Akka Streams and its amazing handling of stream back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially use cases where the amount of work increases as you process make you really value the back-pressure.
This talk takes a sample web crawler use case where each processing pass expands to a larger and larger workload to process, and discusses how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
In addition, we will also provide some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Akka Streams is an implementation of Reactive Streams, which is a standard for asynchronous stream processing with non-blocking backpressure on the JVM. In this talk we'll cover the rationale behind Reactive Streams, and explore the different building blocks available in Akka Streams. I'll use Scala for all coding examples, but Akka Streams also provides a full-fledged Java8 API.After this session you will be all set and ready to reap the benefits of using Akka Streams!
Akka Streams are an implementation of the Reactive Streams specification (http://reactive-streams.org/), a joint effort that aims at standardizing the exchange of streams of data across asynchronous boundaries in a fully non-blocking way while providing flow control and mediating back pressure. In this presentation we go into the details of what this new abstraction can be used for and what the guiding principles are behind its development. We then focus on one prominent use-case which is the upcoming Akka HTTP module: a fully stream-enabled, reactive HTTP server and client implementation.
Akka Streams is a toolkit for processing of streams. It is an implementation of Reactive Streams Specification. Its purpose is to “formulate stream processing setups such that we can then execute them efficiently and with bounded resource usage.”
Journey into Reactive Streams and Akka StreamsKevin Webber
Are streams just collections? What's the difference between Java 8 streams and Reactive Streams? How do I implement Reactive Streams with Akka? Pub/sub, dynamic push/pull, non-blocking, non-dropping; these are some of the other concepts covered. We'll also discuss how to leverage streams in a real-world application.
Akka Streams (0.7) talk for the Tokyo Scala User Group, hosted by Dwango.
Akka streams are an reactive streams implementation which allows for asynchronous back-pressured processing of data in complext pipelines. This talk aims to highlight the details about how reactive streams work as well as some of the ideas behind akka streams.
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Ka...Reactivesummit
Akka Streams and its amazing handling of stream back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially use cases where the amount of work increases as you process make you really value the back-pressure.
This talk takes a sample web crawler use case where each processing pass expands to a larger and larger workload to process, and discusses how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
In addition, we will also provide some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Real-time streaming and data pipelines with Apache KafkaJoe Stein
Get up and running quickly with Apache Kafka http://kafka.apache.org/
* Fast * A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients.
* Scalable * Kafka is designed to allow a single cluster to serve as the central data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of co-ordinated consumers
* Durable * Messages are persisted on disk and replicated within the cluster to prevent data loss. Each broker can handle terabytes of messages without performance impact.
* Distributed by Design * Kafka has a modern cluster-centric design that offers strong durability and fault-tolerance guarantees.
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...Lightbend
Things were easier when all our data used to be offline, analyzed overnight in batches. Now our data is online, in motion, and generated constantly. For architects, developers and their businesses, this means that there is an urgent need for tools and applications that can deliver real-time (or near real-time) streaming ETL capabilities.
In this session by Konrad Malawski, author, speaker and Senior Akka Engineer at Lightbend, you will learn how to build these streaming ETL pipelines with Akka Streams, Alpakka and Apache Kafka, and why they matter to enterprises that are increasingly turning to streaming Fast Data applications.
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & KafkaAkara Sucharitakul
Akka Streams and its amazing handling of stream back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially use cases where the amount of work increases as you process make you really value the back-pressure.
This talk takes a sample web crawler use case where each processing pass expands to a larger and larger workload to process, and discusses how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
In addition, we will also provide some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Service Stampede: Surviving a Thousand ServicesAnil Gursel
How many services do you have? 5, 10, 100? How do you even run large number of services? A micro service may be relatively simple. But services also mean distributed systems, which are inherently complex. 5 services are complex. A thousand services across many generations are at least 200 times as complex. How do we deal with such complexity?
This talk discusses service architecture at Internet scale, the need for larger transaction density, larger horizontal and vertical scale, more predictable latencies under stress, and the need for standardization and visibility. We’ll dive into how we build our latest generation service infrastructure based on Scala and Akka to serve the needs of such a large scale ecosystem.
Lastly, have the cake and eat it too. No, we’re not keeping all the goodies only to ourselves. They are all there for you in open source.
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...Dan Halperin
Apache Beam (incubating) is a unified batch and streaming data processing programming model that is efficient and portable. Beam evolved from a decade of system-building at Google, and Beam pipelines run today on both open source (Apache Flink, Apache Spark) and proprietary (Google Cloud Dataflow) runners. This talk will focus on I/O and connectors in Apache Beam, specifically its APIs for efficient, parallel, adaptive I/O. Google will discuss how these APIs enable a Beam data processing pipeline runner to dynamically rebalance work at runtime, to work around stragglers, and to automatically scale up and down cluster size as a job’s workload changes. Together these APIs and techniques enable Apache Beam runners to efficiently use computing resources without compromising on performance or correctness. Practical examples and a demonstration of Beam will be included.
Stream processing from single node to a clusterGal Marder
Building data pipelines shouldn't be so hard, you just need to choose the right tools for the task.
We will review Akka and Spark streaming, how they work and how to use them and when.
Distributed Systems Done Right: Why Java Enterprises Are Embracing The Actor ...Lightbend
Most likely, your job is heavily focused on helping your organization modernize for the digital era. As the days of purely Object-Oriented Programming and related frameworks come to a close, enterprises migrating to distributed, cloud infrastructures are embracing a different approach: the Actor Model.
When it comes to distributed computing, the Actor Model is the great-grandparent of it all.
Created by Carl Hewitt in 1973, Forrester Research notes, “the Actor model is seeing renewed interest as cloud concurrency challenges grow.”
Yet even if you understand the Actor Model and used some of the toolkits for it (e.g. Akka and Erlang), how do you easily explain the concept to your team, colleagues and managers? Where do you start?
In this webinar, Hugh McKee, Global Solutions Architect at Lightbend, shows you how Actors behave and interact as humans do when it comes to communicating, and how these similar behavioral patterns provide basic intuition when designing Reactive systems. Actors allow your teams to focus on an application’s business logic rather than on low-level protocols, accelerating time-to-market while keeping your infrastructure costs low.
Our goal is twofold: provide you with a comprehensive review of the Actor Model, and give you the resources you need to help others learn why enterprises like Walmart, Intel, Samsung, IBM, Norwegian Cruise Lines and HSBC are committed production users of Akka, the JVM-based toolkit built on the Actor Model.
In this webinar, you’ll learn:
*Why actor-based systems are one of the foundational technologies for creating microservices architecture (MSA)
*How Actors delegate work by creating other Actors in a supervisor-to-worker relationship
*How Actors manage requests and scale horizontally in large systems
*The difference between traditional systems and actor-based systems
*How an Actor system forms clusters when the flow of work exceeds a system’s capacity to process it
*Why failure detection and failure recovery is an architectural feature of Actor systems
*An example of using Actors to build an Internet of Things (IoT) application
Visit Lightbend.com/blog for more goodness.
Slides from my madlab presentation on Akka Streams & Reactive Kafka (October 2015), full slides and source here:
https://github.com/markglh/AkkaStreams-Madlab-Slides
Real-time streaming and data pipelines with Apache KafkaJoe Stein
Get up and running quickly with Apache Kafka http://kafka.apache.org/
* Fast * A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients.
* Scalable * Kafka is designed to allow a single cluster to serve as the central data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of co-ordinated consumers
* Durable * Messages are persisted on disk and replicated within the cluster to prevent data loss. Each broker can handle terabytes of messages without performance impact.
* Distributed by Design * Kafka has a modern cluster-centric design that offers strong durability and fault-tolerance guarantees.
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...Lightbend
Things were easier when all our data used to be offline, analyzed overnight in batches. Now our data is online, in motion, and generated constantly. For architects, developers and their businesses, this means that there is an urgent need for tools and applications that can deliver real-time (or near real-time) streaming ETL capabilities.
In this session by Konrad Malawski, author, speaker and Senior Akka Engineer at Lightbend, you will learn how to build these streaming ETL pipelines with Akka Streams, Alpakka and Apache Kafka, and why they matter to enterprises that are increasingly turning to streaming Fast Data applications.
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & KafkaAkara Sucharitakul
Akka Streams and its amazing handling of stream back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially use cases where the amount of work increases as you process make you really value the back-pressure.
This talk takes a sample web crawler use case where each processing pass expands to a larger and larger workload to process, and discusses how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
In addition, we will also provide some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Service Stampede: Surviving a Thousand ServicesAnil Gursel
How many services do you have? 5, 10, 100? How do you even run large number of services? A micro service may be relatively simple. But services also mean distributed systems, which are inherently complex. 5 services are complex. A thousand services across many generations are at least 200 times as complex. How do we deal with such complexity?
This talk discusses service architecture at Internet scale, the need for larger transaction density, larger horizontal and vertical scale, more predictable latencies under stress, and the need for standardization and visibility. We’ll dive into how we build our latest generation service infrastructure based on Scala and Akka to serve the needs of such a large scale ecosystem.
Lastly, have the cake and eat it too. No, we’re not keeping all the goodies only to ourselves. They are all there for you in open source.
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...Dan Halperin
Apache Beam (incubating) is a unified batch and streaming data processing programming model that is efficient and portable. Beam evolved from a decade of system-building at Google, and Beam pipelines run today on both open source (Apache Flink, Apache Spark) and proprietary (Google Cloud Dataflow) runners. This talk will focus on I/O and connectors in Apache Beam, specifically its APIs for efficient, parallel, adaptive I/O. Google will discuss how these APIs enable a Beam data processing pipeline runner to dynamically rebalance work at runtime, to work around stragglers, and to automatically scale up and down cluster size as a job’s workload changes. Together these APIs and techniques enable Apache Beam runners to efficiently use computing resources without compromising on performance or correctness. Practical examples and a demonstration of Beam will be included.
Stream processing from single node to a clusterGal Marder
Building data pipelines shouldn't be so hard, you just need to choose the right tools for the task.
We will review Akka and Spark streaming, how they work and how to use them and when.
Distributed Systems Done Right: Why Java Enterprises Are Embracing The Actor ...Lightbend
Most likely, your job is heavily focused on helping your organization modernize for the digital era. As the days of purely Object-Oriented Programming and related frameworks come to a close, enterprises migrating to distributed, cloud infrastructures are embracing a different approach: the Actor Model.
When it comes to distributed computing, the Actor Model is the great-grandparent of it all.
Created by Carl Hewitt in 1973, Forrester Research notes, “the Actor model is seeing renewed interest as cloud concurrency challenges grow.”
Yet even if you understand the Actor Model and used some of the toolkits for it (e.g. Akka and Erlang), how do you easily explain the concept to your team, colleagues and managers? Where do you start?
In this webinar, Hugh McKee, Global Solutions Architect at Lightbend, shows you how Actors behave and interact as humans do when it comes to communicating, and how these similar behavioral patterns provide basic intuition when designing Reactive systems. Actors allow your teams to focus on an application’s business logic rather than on low-level protocols, accelerating time-to-market while keeping your infrastructure costs low.
Our goal is twofold: provide you with a comprehensive review of the Actor Model, and give you the resources you need to help others learn why enterprises like Walmart, Intel, Samsung, IBM, Norwegian Cruise Lines and HSBC are committed production users of Akka, the JVM-based toolkit built on the Actor Model.
In this webinar, you’ll learn:
*Why actor-based systems are one of the foundational technologies for creating microservices architecture (MSA)
*How Actors delegate work by creating other Actors in a supervisor-to-worker relationship
*How Actors manage requests and scale horizontally in large systems
*The difference between traditional systems and actor-based systems
*How an Actor system forms clusters when the flow of work exceeds a system’s capacity to process it
*Why failure detection and failure recovery is an architectural feature of Actor systems
*An example of using Actors to build an Internet of Things (IoT) application
Visit Lightbend.com/blog for more goodness.
Slides from my madlab presentation on Akka Streams & Reactive Kafka (October 2015), full slides and source here:
https://github.com/markglh/AkkaStreams-Madlab-Slides
Managing Binary Compatibility in Scala (Scala Days 2011)mircodotta
The following is the abstract submitted for my Scala Days 2011 talk:
Binary compatibility is not a topic specific to the Scala language, but rather a concern for all languages targeting the JVM, Java included. Scala shares with Java many sources of potential binary incompatibilities, however, because of Scala greater expressiveness, Scala code has unique sources of incompatibility.
The Scala programming language offers several language constructs that do not have an equivalent in Java and are not natively supported by the JVM. Because of this, the Scala compiler (scalac) transforms these constructs into lower-lever, Java compatible, patterns that can be then easily translated into bytecode. Good examples of such high-level Scala constructs are traits, for mixin-based inheritance, and functions as first data citizens.
During this presentation we will review the main sources of binary incompatibility for the Scala language, providing you with useful insights about how you should evolve your codebase to avoid binary incompatibilities. Furthermore, we will show a tool, the Migration Manager, that can be used to automatically diagnose binary incompatibilities between two versions of a same library.
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...mircodotta
The demands and expectations for applications have changed dramatically in recent years. Applications today are deployed on a wide range of infrastructure; from mobile devices up to thousands of nodes running in the cloud—all powered by multi-core processors. They need to be rich and collaborative, have a real-time feel with millisecond response time and should never stop running. Additionally, modern applications are a mashup of external services that need to be consumed and composed to provide the features at hand.
We are seeing a new type of applications emerging to address these new challenges—these are being called Reactive Applications. In this talk we will discuss four key traits of Reactive; Event-Driven, Scalable, Resilient and Responsive—how they impact application design, how they interact, their supporting technologies and techniques, how to think when designing and building them—all to make it easier for you and your team to Go Reactive.
(All credit for this great slidedeck goes to @rolandkuhn, while any inaccuracy is solely mine. The session was not recorded, but if you find the content interesting, you should definitely watch Roland Kuhn presenting it http://www.infoq.com/presentations/reactive-principles)
Can you believe that the Scala programming language is already 13 years old? Scala was an experiment back in 2003, but there are no questions today about its success and its great influence on other languages, especially on Java. In this session we will travel time, going back to the age when Java was the hacker’s drink, while Pizza was the hacker’s food. We will glance through some of the memorable moments, and land in the present days to introduce all the goodnesses available in the upcoming Scala 2.12 release. Finally, we will take a brief but intense look at what we can expect from the future. Prepare your time traveling equipment, and be ready to rewind the clock to more than 20 years ago!
Enterprise Development Trends 2016 - Cloud, Container and Microservices Insig...Lightbend
In the past, infrastructure was left to operations teams. Today, it’s JVM developers themselves being brought into the DevOps tent based on the new characteristics of the modern enterprise application, as well as major innovations in the infrastructure running it. Lightbend surveyed 2,151 global respondents working on the JVM to discover:
Correlations between development trends and IT infrastructure trends
How organizations at the forefront of digital transformation are modernizing their applications
Real production usage break-downs of today’s most buzzed about emerging technologies
The survey gathered responses from a diverse range of companies, with 20 percent of respondents hailing from companies with more than 5,000 employees (large organizations), 28 percent from companies with 200-5,000 employees (medium sized organizations) and 52 percent from companies with fewer than 200 employees.
Through the lens of the Striving Styles, each of the Myers-Briggs Type Indicator® (MBTI®) 16 Types comes alive. Discover which quadrant of the brain the INTP prefers to use and the predominant, innate psychological need that drives the INTP's behavior as well as how an INTP can leverage their whole brain in order to develop and achieve their potential.
The SSPS shows that each of the four quadrants of the brain has their own role, or function, to play in our personality and our consciousness. Based on Jung's Theory of Psychological Type, we are hard-wired to prefer one function or quadrant over the others and we use each quadrant or function in either an inwardly or an outwardly way.
Social Media for Assisted Living: Best Friend or Worst Enemy?Laura Click
Social media can be a powerful tool for assisted living facilities to market and promote their services, especially since these organizations rely heavily on referrals for new business.
This presentation offers up some tips for how assisted living organizations can use social media to engage with their residents, families and even prospects.
This also covers some of the pitfalls and downsides to social media, how to handle it when it happens and how to prevent it in the future.
Akka Streams is an implementation of Reactive Streams, which is a standard for asynchronous stream processing with non-blocking backpressure on the JVM. In this talk we'll cover the rationale behind Reactive Streams, and explore the different building blocks available in Akka Streams. I'll use Scala for all coding examples, but Akka Streams also provides a full-fledged Java8 API. After this session you will be all set and ready to reap the benefits of using Akka Streams!
In this presentation, Akka Team Lead and author Roland Kuhn presents the freshly released final specification for Reactive Streams on the JVM. This work was done in collaboration with engineers representing Netflix, Red Hat, Pivotal, Oracle, Typesafe and others to define a standard for passing streams of data between threads in an asynchronous and non-blocking fashion. This is a common need in Reactive systems, where handling streams of "live" data whose volume is not predetermined.
The most prominent issue facing the industry today is that resource consumption needs to be controlled such that a fast data source does not overwhelm the stream destination. Asynchrony is needed in order to enable the parallel use of computing resources, on collaborating network hosts or multiple CPU cores within a single machine.
Here we'll review the mechanisms employed by Reactive Streams, discuss the applicability of this technology to a variety of problems encountered in day to day work on the JVM, and give an overview of the tooling ecosystem that is emerging around this young standard.
Reactive Streams 1.0.0 is now live, and so are our implementations in Akka Streams 1.0 and Slick 3.0.
Reactive Streams is an engineering collaboration between heavy hitters in the area of streaming data on the JVM. With the Reactive Streams Special Interest Group, we set out to standardize a common ground for achieving statically-typed, high-performance, low latency, asynchronous streams of data with built-in non-blocking back pressure—with the goal of creating a vibrant ecosystem of interoperating implementations, and with a vision of one day making it into a future version of Java.
Akka (recent winner of “Most Innovative Open Source Tech in 2015”) is a toolkit for building message-driven applications. With Akka Streams 1.0, Akka has incorporated a graphical DSL for composing data streams, an execution model that decouples the stream’s staged computation—it’s “blueprint”—from its execution (allowing for actor-based, single-threaded and fully distributed and clustered execution), type safe stream composition, an implementation of the Reactive Streaming specification that enables back-pressure, and more than 20 predefined stream “processing stages” that provide common streaming transformations that developers can tap into (for splitting streams, transforming streams, merging streams, and more).
Slick is a relational database query and access library for Scala that enables loose-coupling, minimal configuration requirements and abstraction of the complexities of connecting with relational databases. With Slick 3.0, Slick now supports the Reactive Streams API for providing asynchronous stream processing with non-blocking back-pressure. Slick 3.0 also allows elegant mapping across multiple data types, static verification and type inference for embedded SQL statements, compile-time error discovery, and JDBC support for interoperability with all existing drivers.
With the advent of “big data”, it has become inevitable to analyze huge volumes of data in real-time to make sense out of it. For this to happen seamlessly, the streaming of that data is necessary. This is where Reactive Streams step in.
Akka Streams is built on top of the Reactive Streams interface. This webinar will be an introduction to Akka Streams and how it simplifies the aspect of back-pressure in real-time streaming.
Here’s an outline of the webinar -
~ Introduction to the problem set
~ How do Akka Streams help simplify the problem of back-pressure?
~ Basic terminologies of Akka Streams
~ Live demo of a real-life problem being solved with Akka Streams
Everyone in the Scala world is using or looking into using Akka for low-latency, scalable, distributed or concurrent systems. I'd like to share my story of developing and productionizing multiple Akka apps, including low-latency ingestion and real-time processing systems, and Spark-based applications.
When does one use actors vs futures?
Can we use Akka with, or in place of, Storm?
How did we set up instrumentation and monitoring in production?
How does one use VisualVM to debug Akka apps in production?
What happens if the mailbox gets full?
What is our Akka stack like?
I will share best practices for building Akka and Scala apps, pitfalls and things we'd like to avoid, and a vision of where we would like to go for ideal Akka monitoring, instrumentation, and debugging facilities. Plus backpressure and at-least-once processing.
Learn from HomeAway Hadoop Development and Operations Best PracticesDriven Inc.
HomeAway's Big Data team shares a number of development best practices using Cascading, a data application development framework. They also review several operational best practices for managing production Big Data applications that are business critical.
Kafka is primarily used to build real-time streaming data pipelines and applications that adapt to the data streams. It combines messaging, storage, and stream processing to allow storage and analysis of both historical and real-time data.
SFBigAnalytics_20190724: Monitor kafka like a ProChester Chen
Kafka operators need to provide guarantees to the business that Kafka is working properly and delivering data in real time, and they need to identify and triage problems so they can solve them before end users notice them. This elevates the importance of Kafka monitoring from a nice-to-have to an operational necessity. In this talk, Kafka operations experts Xavier Léauté and Gwen Shapira share their best practices for monitoring Kafka and the streams of events flowing through it. How to detect duplicates, catch buggy clients, and triage performance issues – in short, how to keep the business’s central nervous system healthy and humming along, all like a Kafka pro.
Speakers: Gwen Shapira, Xavier Leaute (Confluence)
Gwen is a software engineer at Confluent working on core Apache Kafka. She has 15 years of experience working with code and customers to build scalable data architectures. She currently specializes in building real-time reliable data processing pipelines using Apache Kafka. Gwen is an author of “Kafka - the Definitive Guide”, "Hadoop Application Architectures", and a frequent presenter at industry conferences. Gwen is also a committer on the Apache Kafka and Apache Sqoop projects.
Xavier Leaute is One of the first engineers to Confluent team, Xavier is responsible for analytics infrastructure, including real-time analytics in KafkaStreams. He was previously a quantitative researcher at BlackRock. Prior to that, he held various research and analytics roles at Barclays Global Investors and MSCI.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
4. Why Reactive?
• Users expectations have changed
• Services must be always up.
• Must be fast.
• Billions of internet connected devices.
• Data is transformed and pushed continuously.
4
5. Reactive Streams
An initiative for providing
Standardised(!)
Back-pressured
Asynchronous
Stream processing
!
http://www.reactive-streams.org/
5
13. Akka Streams: Basics
• DSL for the formulation of transformations on
data streams.
• Basic building blocks:
• Source
-‐
something with exactly one output stream.
• Flow - something with exactly one input and one output
stream.
• Sink - something with exactly one input stream.
• RunnableFlow - A Flow that has both ends “attached”
to a Source and Sink respectively, and is ready to be run() .
13
20. Akka Streams: Fan-out
• Broadcast
-‐
given an input element emits to each
output.
• Balance- given an input element emits to one of its
output ports.
• UnZip - splits a stream of (A,B) tuples into two
streams, one of type A and on of type B.
• FlexiRoute
-‐ enables writing custom fan out
elements using a simple DSL.
20
21. Akka Streams: Fan-in
• Merge
-‐
picks randomly from inputs pushing them one
by one to its output.
• MergePreferred
- like Merge but if elements are
available on preferred port, it picks from it, otherwise
randomly from others.
•
ZipWith(fn)- takes a function of N inputs that
given a value for each input emits 1 output element.
21
22. Akka Streams: Fan-in cont’d
• Zip
- is a ZipWith specialised to zipping input streams
of A and B into an (A,B) tuple stream.
• Concat
- concatenates two streams (first consume
one, then the second one).
• FlexiMerge
- enables writing custom fan-in
elements using a simple DSL.
22
54. Is that really all there is to know?
• Naaaa, there is a lot more for you to explore!
• If the existing building blocks are not enough, define
your owns.
• Use mapAsync/mapAsyncUnordered for
integrating with external services.
• Streams Error Handling.
• Handling TCP connections with Streams.
• Integration with Actors.
54
55. What now?
• Use it:
"com.typesafe.akka" %% "akka-stream-experimental" % "1.0-RC2"
• Check out the Activator template
Akka Streams with Java8orScala.
• Akka Streams API doc and user guide for both
Java8 and Scala.
• Code used for the demos https://github.com/
dotta/akka-streams-demo/releases/tag/v02
55
56. Next Steps
• Akka Streams 1.0 final soon.
• Inclusion in future JDK (shooting for JDK9)
• We aim at polyglot standard (JS, wire proto)
• Try it out and give feedback!
• http://reactive-streams.org/
• https://github.com/reactive-streams
56
