Akka Streams is an implementation of Reactive Streams, which is a standard for asynchronous stream processing with non-blocking backpressure on the JVM. In this talk we'll cover the rationale behind Reactive Streams, and explore the different building blocks available in Akka Streams. I'll use Scala for all coding examples, but Akka Streams also provides a full-fledged Java8 API.After this session you will be all set and ready to reap the benefits of using Akka Streams!
Akka Streams is a toolkit for processing of streams. It is an implementation of Reactive Streams Specification. Its purpose is to “formulate stream processing setups such that we can then execute them efficiently and with bounded resource usage.”
Akka Streams is an implementation of Reactive Streams, which is a standard for asynchronous stream processing with non-blocking backpressure on the JVM. In this talk we'll cover the rationale behind Reactive Streams, and explore the different building blocks available in Akka Streams. I'll use Scala for all coding examples, but Akka Streams also provides a full-fledged Java8 API.After this session you will be all set and ready to reap the benefits of using Akka Streams!
Akka Streams are an implementation of the Reactive Streams specification (http://reactive-streams.org/), a joint effort that aims at standardizing the exchange of streams of data across asynchronous boundaries in a fully non-blocking way while providing flow control and mediating back pressure. In this presentation we go into the details of what this new abstraction can be used for and what the guiding principles are behind its development. We then focus on one prominent use-case which is the upcoming Akka HTTP module: a fully stream-enabled, reactive HTTP server and client implementation.
Journey into Reactive Streams and Akka StreamsKevin Webber
Are streams just collections? What's the difference between Java 8 streams and Reactive Streams? How do I implement Reactive Streams with Akka? Pub/sub, dynamic push/pull, non-blocking, non-dropping; these are some of the other concepts covered. We'll also discuss how to leverage streams in a real-world application.
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Ka...Reactivesummit
Akka Streams and its amazing handling of stream back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially use cases where the amount of work increases as you process make you really value the back-pressure.
This talk takes a sample web crawler use case where each processing pass expands to a larger and larger workload to process, and discusses how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
In addition, we will also provide some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Akka Streams is a toolkit for processing of streams. It is an implementation of Reactive Streams Specification. Its purpose is to “formulate stream processing setups such that we can then execute them efficiently and with bounded resource usage.”
Akka Streams is an implementation of Reactive Streams, which is a standard for asynchronous stream processing with non-blocking backpressure on the JVM. In this talk we'll cover the rationale behind Reactive Streams, and explore the different building blocks available in Akka Streams. I'll use Scala for all coding examples, but Akka Streams also provides a full-fledged Java8 API.After this session you will be all set and ready to reap the benefits of using Akka Streams!
Akka Streams are an implementation of the Reactive Streams specification (http://reactive-streams.org/), a joint effort that aims at standardizing the exchange of streams of data across asynchronous boundaries in a fully non-blocking way while providing flow control and mediating back pressure. In this presentation we go into the details of what this new abstraction can be used for and what the guiding principles are behind its development. We then focus on one prominent use-case which is the upcoming Akka HTTP module: a fully stream-enabled, reactive HTTP server and client implementation.
Journey into Reactive Streams and Akka StreamsKevin Webber
Are streams just collections? What's the difference between Java 8 streams and Reactive Streams? How do I implement Reactive Streams with Akka? Pub/sub, dynamic push/pull, non-blocking, non-dropping; these are some of the other concepts covered. We'll also discuss how to leverage streams in a real-world application.
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Ka...Reactivesummit
Akka Streams and its amazing handling of stream back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially use cases where the amount of work increases as you process make you really value the back-pressure.
This talk takes a sample web crawler use case where each processing pass expands to a larger and larger workload to process, and discusses how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
In addition, we will also provide some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...Lightbend
Things were easier when all our data used to be offline, analyzed overnight in batches. Now our data is online, in motion, and generated constantly. For architects, developers and their businesses, this means that there is an urgent need for tools and applications that can deliver real-time (or near real-time) streaming ETL capabilities.
In this session by Konrad Malawski, author, speaker and Senior Akka Engineer at Lightbend, you will learn how to build these streaming ETL pipelines with Akka Streams, Alpakka and Apache Kafka, and why they matter to enterprises that are increasingly turning to streaming Fast Data applications.
Akka Streams (0.7) talk for the Tokyo Scala User Group, hosted by Dwango.
Akka streams are an reactive streams implementation which allows for asynchronous back-pressured processing of data in complext pipelines. This talk aims to highlight the details about how reactive streams work as well as some of the ideas behind akka streams.
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & KafkaAkara Sucharitakul
Akka Streams and its amazing handling of stream back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially use cases where the amount of work increases as you process make you really value the back-pressure.
This talk takes a sample web crawler use case where each processing pass expands to a larger and larger workload to process, and discusses how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
In addition, we will also provide some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Real-time streaming and data pipelines with Apache KafkaJoe Stein
Get up and running quickly with Apache Kafka http://kafka.apache.org/
* Fast * A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients.
* Scalable * Kafka is designed to allow a single cluster to serve as the central data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of co-ordinated consumers
* Durable * Messages are persisted on disk and replicated within the cluster to prevent data loss. Each broker can handle terabytes of messages without performance impact.
* Distributed by Design * Kafka has a modern cluster-centric design that offers strong durability and fault-tolerance guarantees.
Service Stampede: Surviving a Thousand ServicesAnil Gursel
How many services do you have? 5, 10, 100? How do you even run large number of services? A micro service may be relatively simple. But services also mean distributed systems, which are inherently complex. 5 services are complex. A thousand services across many generations are at least 200 times as complex. How do we deal with such complexity?
This talk discusses service architecture at Internet scale, the need for larger transaction density, larger horizontal and vertical scale, more predictable latencies under stress, and the need for standardization and visibility. We’ll dive into how we build our latest generation service infrastructure based on Scala and Akka to serve the needs of such a large scale ecosystem.
Lastly, have the cake and eat it too. No, we’re not keeping all the goodies only to ourselves. They are all there for you in open source.
Reactive Streams: Handling Data-Flow the Reactive WayRoland Kuhn
Building on the success of Reactive Extensions—first in Rx.NET and now in RxJava—we are taking Observers and Observables to the next level: by adding the capability of handling back-pressure between asynchronous execution stages we enable the distribution of stream processing across a cluster of potentially thousands of nodes. The project defines the common interfaces for interoperable stream implementations on the JVM and is the result of a collaboration between Twitter, Netflix, Pivotal, RedHat and Typesafe. In this presentation I introduce the guiding principles behind its design and show examples using the actor-based implementation in Akka.
Developing Secure Scala Applications With Fortify For ScalaLightbend
From banks to airlines to credit rating agencies, security continues to be a major focus for organizations across various industries. As the newspapers show, it’s heavily damaging to enterprises when security vulnerabilities in their code, infrastructure, or open source frameworks/libraries get exploited.
The good news is that your Scala development team now has a powerful ally for securing their applications. Co-developed by the Fortify team along with Lightbend, the upcoming Fortify for Scala Plugin is the only Static Application Security Testing (SAST) solution to use the official Scala compiler. This plugin automatically identifies code-level security vulnerabilities early in the SDLC, so you can confidently and reliably secure your mission-critical Scala-based applications.
In this webinar by Seth Tisue, Scala Committer and Senior Scala Engineer at Lightbend, and Poonam Yadav, Product Manager for Fortify at Micro Focus, you will learn about:
* Some of the more than 200 vulnerabilities that the Fortify plugin for Scala can catch and help you resolve,
* How the plugin works to analyze, identify and provide actionable recommendations,
* How to integrate it into your modern DevOps environment,
* Why this plugin was co-developed by Lightbend and the Fortify team, and how it benefits your organization’s security professionals / CISO office.
Revitalizing Enterprise Integration with Reactive StreamsLightbend
With Viktor Klang, Deputy CTO Lightbend, Inc.
As software grows more and more interconnected, and with several generations of software having to interoperate, a new take on the integration of systems is needed—ad hoc, unversioned, and unreplicated scripts just won’t suffice, and the traditional Enterprise Service Bus (ESB) concept has experienced stability, reliability, performance, and scalability problems.
In this webinar, Viktor explores a new take on Enterprise Integration Patterns:
First, he will explore the Reactive Streams standard, an orchestration layer where transformations are standalone, composable, reusable, and—most importantly—using asynchronous flow-control—back pressure—to maintain predictable, stable, behavior over time.
Furthermore, he will go through how one-off workloads relate to continuous, and batch, workloads, and how they can be addressed by that very same orchestration layer.
Finally, he will review how this type of design achieves resilience, scalability, and ultimately—responsiveness.
Stream processing from single node to a clusterGal Marder
Building data pipelines shouldn't be so hard, you just need to choose the right tools for the task.
We will review Akka and Spark streaming, how they work and how to use them and when.
A dive into akka streams: from the basics to a real-world scenarioGioia Ballin
Reactive streaming is becoming the best approach to handle data flows across asynchronous boundaries. Here, we present the implementation of a real-world application based on Akka Streams. After reviewing the basics, we will discuss the development of a data processing pipeline that collects real-time sensor data and sends it to a Kinesis stream. There are various possible point of failures in this architecture. What should happen when Kinesis is unavailable? If the data flow is not handled in the correct way, some information may get lost. Akka Streams are the tools that enabled us to build a reliable processing logic for the pipeline that avoids data losses and maximizes the robustness of the entire system.
Reactive Streams are a cross-company initiative first ignited by Lightbend in 2013, soon to be joined by RxJava and other implementations focused on solving a very similar problem: asynchronous non-blocking stream processing, with guaranteed over-flow protection. Fast forward to 2016 and now these interfaces are part of JSR-266 and proposed for JDK9.
In this talk we'll first disambiguate what the word Stream means in this context (as it's been overloaded recently by various different meanings), then look at how its protocol works and how one might use it in the real world showing examples using existing implementations.
We'll also have a peek into the future, to see what the next steps for such collaborative protocols and the JDK ecosystem are in general.
In this presentation, Akka Team Lead and author Roland Kuhn presents the freshly released final specification for Reactive Streams on the JVM. This work was done in collaboration with engineers representing Netflix, Red Hat, Pivotal, Oracle, Typesafe and others to define a standard for passing streams of data between threads in an asynchronous and non-blocking fashion. This is a common need in Reactive systems, where handling streams of "live" data whose volume is not predetermined.
The most prominent issue facing the industry today is that resource consumption needs to be controlled such that a fast data source does not overwhelm the stream destination. Asynchrony is needed in order to enable the parallel use of computing resources, on collaborating network hosts or multiple CPU cores within a single machine.
Here we'll review the mechanisms employed by Reactive Streams, discuss the applicability of this technology to a variety of problems encountered in day to day work on the JVM, and give an overview of the tooling ecosystem that is emerging around this young standard.
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...Lightbend
Things were easier when all our data used to be offline, analyzed overnight in batches. Now our data is online, in motion, and generated constantly. For architects, developers and their businesses, this means that there is an urgent need for tools and applications that can deliver real-time (or near real-time) streaming ETL capabilities.
In this session by Konrad Malawski, author, speaker and Senior Akka Engineer at Lightbend, you will learn how to build these streaming ETL pipelines with Akka Streams, Alpakka and Apache Kafka, and why they matter to enterprises that are increasingly turning to streaming Fast Data applications.
Akka Streams (0.7) talk for the Tokyo Scala User Group, hosted by Dwango.
Akka streams are an reactive streams implementation which allows for asynchronous back-pressured processing of data in complext pipelines. This talk aims to highlight the details about how reactive streams work as well as some of the ideas behind akka streams.
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & KafkaAkara Sucharitakul
Akka Streams and its amazing handling of stream back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially use cases where the amount of work increases as you process make you really value the back-pressure.
This talk takes a sample web crawler use case where each processing pass expands to a larger and larger workload to process, and discusses how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
In addition, we will also provide some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Real-time streaming and data pipelines with Apache KafkaJoe Stein
Get up and running quickly with Apache Kafka http://kafka.apache.org/
* Fast * A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients.
* Scalable * Kafka is designed to allow a single cluster to serve as the central data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of co-ordinated consumers
* Durable * Messages are persisted on disk and replicated within the cluster to prevent data loss. Each broker can handle terabytes of messages without performance impact.
* Distributed by Design * Kafka has a modern cluster-centric design that offers strong durability and fault-tolerance guarantees.
Service Stampede: Surviving a Thousand ServicesAnil Gursel
How many services do you have? 5, 10, 100? How do you even run large number of services? A micro service may be relatively simple. But services also mean distributed systems, which are inherently complex. 5 services are complex. A thousand services across many generations are at least 200 times as complex. How do we deal with such complexity?
This talk discusses service architecture at Internet scale, the need for larger transaction density, larger horizontal and vertical scale, more predictable latencies under stress, and the need for standardization and visibility. We’ll dive into how we build our latest generation service infrastructure based on Scala and Akka to serve the needs of such a large scale ecosystem.
Lastly, have the cake and eat it too. No, we’re not keeping all the goodies only to ourselves. They are all there for you in open source.
Reactive Streams: Handling Data-Flow the Reactive WayRoland Kuhn
Building on the success of Reactive Extensions—first in Rx.NET and now in RxJava—we are taking Observers and Observables to the next level: by adding the capability of handling back-pressure between asynchronous execution stages we enable the distribution of stream processing across a cluster of potentially thousands of nodes. The project defines the common interfaces for interoperable stream implementations on the JVM and is the result of a collaboration between Twitter, Netflix, Pivotal, RedHat and Typesafe. In this presentation I introduce the guiding principles behind its design and show examples using the actor-based implementation in Akka.
Developing Secure Scala Applications With Fortify For ScalaLightbend
From banks to airlines to credit rating agencies, security continues to be a major focus for organizations across various industries. As the newspapers show, it’s heavily damaging to enterprises when security vulnerabilities in their code, infrastructure, or open source frameworks/libraries get exploited.
The good news is that your Scala development team now has a powerful ally for securing their applications. Co-developed by the Fortify team along with Lightbend, the upcoming Fortify for Scala Plugin is the only Static Application Security Testing (SAST) solution to use the official Scala compiler. This plugin automatically identifies code-level security vulnerabilities early in the SDLC, so you can confidently and reliably secure your mission-critical Scala-based applications.
In this webinar by Seth Tisue, Scala Committer and Senior Scala Engineer at Lightbend, and Poonam Yadav, Product Manager for Fortify at Micro Focus, you will learn about:
* Some of the more than 200 vulnerabilities that the Fortify plugin for Scala can catch and help you resolve,
* How the plugin works to analyze, identify and provide actionable recommendations,
* How to integrate it into your modern DevOps environment,
* Why this plugin was co-developed by Lightbend and the Fortify team, and how it benefits your organization’s security professionals / CISO office.
Revitalizing Enterprise Integration with Reactive StreamsLightbend
With Viktor Klang, Deputy CTO Lightbend, Inc.
As software grows more and more interconnected, and with several generations of software having to interoperate, a new take on the integration of systems is needed—ad hoc, unversioned, and unreplicated scripts just won’t suffice, and the traditional Enterprise Service Bus (ESB) concept has experienced stability, reliability, performance, and scalability problems.
In this webinar, Viktor explores a new take on Enterprise Integration Patterns:
First, he will explore the Reactive Streams standard, an orchestration layer where transformations are standalone, composable, reusable, and—most importantly—using asynchronous flow-control—back pressure—to maintain predictable, stable, behavior over time.
Furthermore, he will go through how one-off workloads relate to continuous, and batch, workloads, and how they can be addressed by that very same orchestration layer.
Finally, he will review how this type of design achieves resilience, scalability, and ultimately—responsiveness.
Stream processing from single node to a clusterGal Marder
Building data pipelines shouldn't be so hard, you just need to choose the right tools for the task.
We will review Akka and Spark streaming, how they work and how to use them and when.
A dive into akka streams: from the basics to a real-world scenarioGioia Ballin
Reactive streaming is becoming the best approach to handle data flows across asynchronous boundaries. Here, we present the implementation of a real-world application based on Akka Streams. After reviewing the basics, we will discuss the development of a data processing pipeline that collects real-time sensor data and sends it to a Kinesis stream. There are various possible point of failures in this architecture. What should happen when Kinesis is unavailable? If the data flow is not handled in the correct way, some information may get lost. Akka Streams are the tools that enabled us to build a reliable processing logic for the pipeline that avoids data losses and maximizes the robustness of the entire system.
Reactive Streams are a cross-company initiative first ignited by Lightbend in 2013, soon to be joined by RxJava and other implementations focused on solving a very similar problem: asynchronous non-blocking stream processing, with guaranteed over-flow protection. Fast forward to 2016 and now these interfaces are part of JSR-266 and proposed for JDK9.
In this talk we'll first disambiguate what the word Stream means in this context (as it's been overloaded recently by various different meanings), then look at how its protocol works and how one might use it in the real world showing examples using existing implementations.
We'll also have a peek into the future, to see what the next steps for such collaborative protocols and the JDK ecosystem are in general.
In this presentation, Akka Team Lead and author Roland Kuhn presents the freshly released final specification for Reactive Streams on the JVM. This work was done in collaboration with engineers representing Netflix, Red Hat, Pivotal, Oracle, Typesafe and others to define a standard for passing streams of data between threads in an asynchronous and non-blocking fashion. This is a common need in Reactive systems, where handling streams of "live" data whose volume is not predetermined.
The most prominent issue facing the industry today is that resource consumption needs to be controlled such that a fast data source does not overwhelm the stream destination. Asynchrony is needed in order to enable the parallel use of computing resources, on collaborating network hosts or multiple CPU cores within a single machine.
Here we'll review the mechanisms employed by Reactive Streams, discuss the applicability of this technology to a variety of problems encountered in day to day work on the JVM, and give an overview of the tooling ecosystem that is emerging around this young standard.
Understanding Akka Streams, Back Pressure, and Asynchronous ArchitecturesLightbend
The term 'streams' has been getting pretty overloaded recently–it's hard to know where to best use different technologies with streams in the name. In this talk by noted hAkker Konrad Malawski, we'll disambiguate what streams are and what they aren't, taking a deeper look into Akka Streams (the implementation) and Reactive Streams (the standard).
You'll be introduced to a number of real life scenarios where applying back-pressure helps to keep your systems fast and healthy at the same time. While the focus is mainly on the Akka Streams implementation, the general principles apply to any kind of asynchronous, message-driven architectures.
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...Lightbend
Akka Streams and its amazing handling of streaming with back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially in use cases where the amount of work continues to increase as you’re processing it. This is where back-pressure really shines.
In this talk for Architects and Dev Managers by Akara Sucharitakul, Principal MTS for Global Platform Frameworks at PayPal, Inc., we look at how back-pressure based on Akka Streams and Kafka is being used at PayPal to handle very bursty workloads.
In addition, Akara will also share experiences in creating a platform based on Akka and Akka Streams that currently processes over 1 billion transactions per day (on just 8 VMs), with the aim of helping teams adopt these technologies. In this webinar, you will:
*Start with a sample web crawler use case to examine what happens when each processing pass expands to a larger and larger workload to process.
*Review how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
*Look at lessons learned, plus some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
Akka-chan's Survival Guide for the Streaming WorldKonrad Malawski
In this talk we dive into the various kinds of "Streaming", what it actually means, where to use which technology and specifically take a look at Akka Streams and their specific use case and strengths.
Managing Binary Compatibility in Scala (Scala Days 2011)mircodotta
The following is the abstract submitted for my Scala Days 2011 talk:
Binary compatibility is not a topic specific to the Scala language, but rather a concern for all languages targeting the JVM, Java included. Scala shares with Java many sources of potential binary incompatibilities, however, because of Scala greater expressiveness, Scala code has unique sources of incompatibility.
The Scala programming language offers several language constructs that do not have an equivalent in Java and are not natively supported by the JVM. Because of this, the Scala compiler (scalac) transforms these constructs into lower-lever, Java compatible, patterns that can be then easily translated into bytecode. Good examples of such high-level Scala constructs are traits, for mixin-based inheritance, and functions as first data citizens.
During this presentation we will review the main sources of binary incompatibility for the Scala language, providing you with useful insights about how you should evolve your codebase to avoid binary incompatibilities. Furthermore, we will show a tool, the Migration Manager, that can be used to automatically diagnose binary incompatibilities between two versions of a same library.
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...mircodotta
The demands and expectations for applications have changed dramatically in recent years. Applications today are deployed on a wide range of infrastructure; from mobile devices up to thousands of nodes running in the cloud—all powered by multi-core processors. They need to be rich and collaborative, have a real-time feel with millisecond response time and should never stop running. Additionally, modern applications are a mashup of external services that need to be consumed and composed to provide the features at hand.
We are seeing a new type of applications emerging to address these new challenges—these are being called Reactive Applications. In this talk we will discuss four key traits of Reactive; Event-Driven, Scalable, Resilient and Responsive—how they impact application design, how they interact, their supporting technologies and techniques, how to think when designing and building them—all to make it easier for you and your team to Go Reactive.
(All credit for this great slidedeck goes to @rolandkuhn, while any inaccuracy is solely mine. The session was not recorded, but if you find the content interesting, you should definitely watch Roland Kuhn presenting it http://www.infoq.com/presentations/reactive-principles)
Can you believe that the Scala programming language is already 13 years old? Scala was an experiment back in 2003, but there are no questions today about its success and its great influence on other languages, especially on Java. In this session we will travel time, going back to the age when Java was the hacker’s drink, while Pizza was the hacker’s food. We will glance through some of the memorable moments, and land in the present days to introduce all the goodnesses available in the upcoming Scala 2.12 release. Finally, we will take a brief but intense look at what we can expect from the future. Prepare your time traveling equipment, and be ready to rewind the clock to more than 20 years ago!
Enterprise Development Trends 2016 - Cloud, Container and Microservices Insig...Lightbend
In the past, infrastructure was left to operations teams. Today, it’s JVM developers themselves being brought into the DevOps tent based on the new characteristics of the modern enterprise application, as well as major innovations in the infrastructure running it. Lightbend surveyed 2,151 global respondents working on the JVM to discover:
Correlations between development trends and IT infrastructure trends
How organizations at the forefront of digital transformation are modernizing their applications
Real production usage break-downs of today’s most buzzed about emerging technologies
The survey gathered responses from a diverse range of companies, with 20 percent of respondents hailing from companies with more than 5,000 employees (large organizations), 28 percent from companies with 200-5,000 employees (medium sized organizations) and 52 percent from companies with fewer than 200 employees.
Distributed Systems Done Right: Why Java Enterprises Are Embracing The Actor ...Lightbend
Most likely, your job is heavily focused on helping your organization modernize for the digital era. As the days of purely Object-Oriented Programming and related frameworks come to a close, enterprises migrating to distributed, cloud infrastructures are embracing a different approach: the Actor Model.
When it comes to distributed computing, the Actor Model is the great-grandparent of it all.
Created by Carl Hewitt in 1973, Forrester Research notes, “the Actor model is seeing renewed interest as cloud concurrency challenges grow.”
Yet even if you understand the Actor Model and used some of the toolkits for it (e.g. Akka and Erlang), how do you easily explain the concept to your team, colleagues and managers? Where do you start?
In this webinar, Hugh McKee, Global Solutions Architect at Lightbend, shows you how Actors behave and interact as humans do when it comes to communicating, and how these similar behavioral patterns provide basic intuition when designing Reactive systems. Actors allow your teams to focus on an application’s business logic rather than on low-level protocols, accelerating time-to-market while keeping your infrastructure costs low.
Our goal is twofold: provide you with a comprehensive review of the Actor Model, and give you the resources you need to help others learn why enterprises like Walmart, Intel, Samsung, IBM, Norwegian Cruise Lines and HSBC are committed production users of Akka, the JVM-based toolkit built on the Actor Model.
In this webinar, you’ll learn:
*Why actor-based systems are one of the foundational technologies for creating microservices architecture (MSA)
*How Actors delegate work by creating other Actors in a supervisor-to-worker relationship
*How Actors manage requests and scale horizontally in large systems
*The difference between traditional systems and actor-based systems
*How an Actor system forms clusters when the flow of work exceeds a system’s capacity to process it
*Why failure detection and failure recovery is an architectural feature of Actor systems
*An example of using Actors to build an Internet of Things (IoT) application
Visit Lightbend.com/blog for more goodness.
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...Konrad Malawski
Japanese subtitles by Yugo Maede-san, thank you very much. Japanese subtitled version of the "How Reactive Streams and Akka Streams change the JVM Ecosystem". http://www.slideshare.net/ktoso/how-reactive-streams-akka-streams-change-the-jvm-ecosystem
Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)mircodotta
Microservices architecture are becoming a de-facto industry standard, but are you satisfied with the current state of the art? We are not, as we believe that building microservices today is more challenging than it should be. Lagom is here to take on this challenge. First, Lagom is opinionated and it will take some of the hard decisions for you, guiding you to produce microservices that adheres to the Reactive tenents. Second, Lagom was built from the ground up around you, the developer, to push your productivity to the next level. If you are familiar with the Play Framework's development environment, imagine that but tuned for building microservices; we are sure you are going to love it! Third, Lagom comes with batteries included for deploying in production: going from development to production could not be easier. In this session, you will get an introduction to the Lightbend Lagom framework. There will be code and live demos to show you in practice how it works and what you can do with it, making you fully equipped to build your next microservices with Lightbend Lagom!
Benefits Of The Actor Model For Cloud Computing: A Pragmatic Overview For Jav...Lightbend
As enterprise development teams increase the time they spend using cloud computing, many are challenged by a move from a scale-up (monolithic) to a scale-out (distributed) architecture. Reactive system development and microservices are two evolving answers that architects are embracing, but making them work well at scale calls for a departure from the traditional approach of object-oriented programming models and defensive programming through try-catch, which is now being replaced by a highly-resilient supervision model and a "let it crash" philosophy.
In this webinar for Architects, guest speaker Jeffrey Hammond, Forrester Vice-President and Principal Analyst joins Jonas Bonér, CTO/Co-founder of Lightbend and creator of Akka, the actor-based, message-driven runtime for the JVM, to discuss one emerging programming pattern that’s gaining popularity with teams developing for the cloud––the Actor model. They will discuss some history, why the Actor model is a better fit for large, scale-out systems and microservices delivery, the types of workloads using it today, and how to implement an Actor-based system in your existing Java environment.
Akka Streams is an implementation of Reactive Streams, which is a standard for asynchronous stream processing with non-blocking backpressure on the JVM. In this talk we'll cover the rationale behind Reactive Streams, and explore the different building blocks available in Akka Streams. I'll use Scala for all coding examples, but Akka Streams also provides a full-fledged Java8 API. After this session you will be all set and ready to reap the benefits of using Akka Streams!
Spark + AI Summit 2019: Apache Spark Listeners: A Crash Course in Fast, Easy ...Landon Robinson
The Spark Listener interface provides a fast, simple and efficient route to monitoring and observing your Spark application - and you can start using it in minutes. In this talk, we'll introduce the Spark Listener interfaces available in core and streaming applications, and show a few ways in which they've changed our world for the better at SpotX. If you're looking for a "Eureka!" moment in monitoring or tracking of your Spark apps, look no further than Spark Listeners and this talk!
Apache Spark Listeners: A Crash Course in Fast, Easy MonitoringDatabricks
The Spark Listener interface provides a fast, simple and efficient route to monitoring and observing your Spark application - and you can start using it in minutes. In this talk, we'll introduce the Spark Listener interfaces available in core and streaming applications, and show a few ways in which they've changed our world for the better at SpotX. If you're looking for a "Eureka!" moment in monitoring or tracking of your Spark apps, look no further than Spark Listeners and this talk!
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...Dan Halperin
Apache Beam (incubating) is a unified batch and streaming data processing programming model that is efficient and portable. Beam evolved from a decade of system-building at Google, and Beam pipelines run today on both open source (Apache Flink, Apache Spark) and proprietary (Google Cloud Dataflow) runners. This talk will focus on I/O and connectors in Apache Beam, specifically its APIs for efficient, parallel, adaptive I/O. Google will discuss how these APIs enable a Beam data processing pipeline runner to dynamically rebalance work at runtime, to work around stragglers, and to automatically scale up and down cluster size as a job’s workload changes. Together these APIs and techniques enable Apache Beam runners to efficiently use computing resources without compromising on performance or correctness. Practical examples and a demonstration of Beam will be included.
Headaches and Breakthroughs in Building Continuous ApplicationsDatabricks
At SpotX, we have built and maintained a portfolio of Spark Streaming applications -- all of which process records in the millions per minute. From pure data ingestion, to ETL, to real-time reporting, to live customer-facing products and features, continuous applications are in our DNA. Come along with us as we outline our journey from square one to present in the world of Spark Streaming. We'll detail what we've learned about efficient processing and monitoring, reliability and stability, and long term support of a streaming app. Come learn from our mistakes, and leave with some handy settings and designs you can implement in your own streaming apps.
Spark + AI Summit 2019: Headaches and Breakthroughs in Building Continuous Ap...Landon Robinson
At SpotX, we have built and maintained a portfolio of Spark Streaming applications -- all of which process records in the millions per minute. From pure data ingestion, to ETL, to real-time reporting, to live customer-facing products and features, continuous applications are in our DNA. Come along with us as we outline our journey from square one to present in the world of Spark Streaming. We'll detail what we've learned about efficient processing and monitoring, reliability and stability, and long term support of a streaming app. Come learn from our mistakes, and leave with some handy settings and designs you can implement in your own streaming apps.
Presented by Landon Robinson and Jack Chapa
Everyone in the Scala world is using or looking into using Akka for low-latency, scalable, distributed or concurrent systems. I'd like to share my story of developing and productionizing multiple Akka apps, including low-latency ingestion and real-time processing systems, and Spark-based applications.
When does one use actors vs futures?
Can we use Akka with, or in place of, Storm?
How did we set up instrumentation and monitoring in production?
How does one use VisualVM to debug Akka apps in production?
What happens if the mailbox gets full?
What is our Akka stack like?
I will share best practices for building Akka and Scala apps, pitfalls and things we'd like to avoid, and a vision of where we would like to go for ideal Akka monitoring, instrumentation, and debugging facilities. Plus backpressure and at-least-once processing.
Stream, Stream, Stream: Different Streaming Methods with Spark and KafkaDataWorks Summit
At NMC (Nielsen Marketing Cloud) we provide our customers (marketers and publishers) real-time analytics tools to profile their target audiences.
To achieve that, we need to ingest billions of events per day into our big data stores, and we need to do it in a scalable yet cost-efficient manner.
In this session, we will discuss how we continuously transform our data infrastructure to support these goals.
Specifically, we will review how we went from CSV files and standalone Java applications all the way to multiple Kafka and Spark clusters, performing a mixture of Streaming and Batch ETLs, and supporting 10x data growth.
We will share our experience as early-adopters of Spark Streaming and Spark Structured Streaming, and how we overcame technical barriers (and there were plenty...).
We will present a rather unique solution of using Kafka to imitate streaming over our Data Lake, while significantly reducing our cloud services' costs.
Topics include :
* Kafka and Spark Streaming for stateless and stateful use-cases
* Spark Structured Streaming as a possible alternative
* Combining Spark Streaming with batch ETLs
* "Streaming" over Data Lake using Kafka
With the advent of “big data”, it has become inevitable to analyze huge volumes of data in real-time to make sense out of it. For this to happen seamlessly, the streaming of that data is necessary. This is where Reactive Streams step in.
Akka Streams is built on top of the Reactive Streams interface. This webinar will be an introduction to Akka Streams and how it simplifies the aspect of back-pressure in real-time streaming.
Here’s an outline of the webinar -
~ Introduction to the problem set
~ How do Akka Streams help simplify the problem of back-pressure?
~ Basic terminologies of Akka Streams
~ Live demo of a real-life problem being solved with Akka Streams
Get ready to experience fast and scalable performance in your web applications as we dive into the world of Reactive Programming. Our guide using WebFlux is perfect for both beginners and experts a like.
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYCKonrad Malawski
Intense 3 hour workshop covering Akka Actors, Cluster, Streams, HTTP and more. Including very advanced patterns.
Presented with Henrik Engstrom at O'Reilly Software Architecture Conference in New York City in 2017
Learn from HomeAway Hadoop Development and Operations Best PracticesDriven Inc.
HomeAway's Big Data team shares a number of development best practices using Cascading, a data application development framework. They also review several operational best practices for managing production Big Data applications that are business critical.
Kafka is a high-throughput, fault-tolerant, scalable platform for building high-volume near-real-time data pipelines. This presentation is about tuning Kafka pipelines for high-performance.
Select configuration parameters and deployment topologies essential to achieve higher throughput and low latency across the pipeline are discussed. Lessons learned in troubleshooting and optimizing a truly global data pipeline that replicates 100GB data under 25 minutes is discussed.
** Video of this talk is here: https://youtu.be/MQGXrrhGUTw **
The first talk of the Meetup on the 11th of April 2017, hosted by weeronline.nl in their Amsterdam offices.
Streams are everywhere! Akka Streams help us model streaming processes using a very descriptive DSL and optimising resource usage.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
5. Why Reactive?
• Users expectations have changed
• Services must be always up.
• Must be fast.
• Billions of internet connected devices.
• Data is transformed and pushed continuously.
6. Reactive Streams
An initiative for providing
Standardised(!)
Back-pressured
Asynchronous
Stream processing
!
http://www.reactive-streams.org/
14. Akka Streams: Basics
• DSL for the formulation of transformations on
data streams.
• Basic building blocks:
• Source
-‐
something with exactly one output stream.
• Flow - something with exactly one input and one output
stream.
• Sink - something with exactly one input stream.
• RunnableFlow - A Flow that has both ends “attached”
to a Source and Sink respectively, and is ready to be run() .
21. Akka Streams: Fan-out
• Broadcast
-‐
given an input element emits to each
output.
• Balance- given an input element emits to one of its
output ports.
• UnZip - splits a stream of (A,B) tuples into two
streams, one of type A and on of type B.
• FlexiRoute
-‐ enables writing custom fan out
elements using a simple DSL.
22. Akka Streams: Fan-in
• Merge
-‐
picks randomly from inputs pushing them one
by one to its output.
• MergePreferred
- like Merge but if elements are
available on preferred port, it picks from it, otherwise
randomly from others.
•
ZipWith(fn)- takes a function of N inputs that
given a value for each input emits 1 output element.
23. Akka Streams: Fan-in cont’d
• Zip
- is a ZipWith specialised to zipping input streams
of A and B into an (A,B) tuple stream.
• Concat
- concatenates two streams (first consume
one, then the second one).
• FlexiMerge
- enables writing custom fan-in
elements using a simple DSL.
55. Is that really all there is to know?
• Naaaa, there is a lot more for you to explore!
• If the existing building blocks are not enough, define
your owns.
• Use mapAsync/mapAsyncUnordered for
integrating with external services.
• Streams Error Handling.
• Handling TCP connections with Streams.
• Integration with Actors.
56. What now?
• Use it:
"com.typesafe.akka" %% "akka-stream-experimental" % "1.0-M5"
• Check out the Activator template
Akka Streams with Java8orScala.
• Akka Streams API doc and user guide for both
Java8 and Scala.
• Code used for the demos https://github.com/
dotta/akka-streams-demo/releases/tag/v01
57. Next Steps
• Akka Streams RC1 soon (before end of April).
• Inclusion in future JDK (shooting for JDK9)
• We aim at polyglot standard (JS, wire proto)
• Try it out and give feedback!
• http://reactive-streams.org/
• https://github.com/reactive-streams
