Slides from http://www.meetup.com/Reactive-Systems-Hamburg/events/232887060
Barys and Simon talked about Akka Cluster. Cluster Sharding allows to transparently distribute work in an Akka cluster with automatic balancing, migration of workers and automatic restart in case of errors. Cluster PubSub offers the publish/subscribe pattern. Akka Distributed Data offers eventually consistent data structures across the cluster, that allow for keeping the cluster's state.
They talked about the Akka Modules and explained how they interplay. Finally, they shared what Risk.Ident have learned running a reactive application based on Akka Cluster in production for almost a year.
Akka Revealed: A JVM Architect's Journey From Resilient Actors To Scalable Cl...Lightbend
By now, you’ve probably heard of Akka, the JVM toolkit for building scalable, resilient and resource efficient applications in Java or Scala. With over 12 open-source and commercial modules in the toolkit, Akka takes developers from actors on a single JVM, all the way out to network partition healing and clusters of servers distributed across fleets of JVMs. But with such a broad range of features, how can Architects and Developers grok Akka from a high-level perspective?
In this technical webinar by Hugh McKee, O’Reilly author and Developer Advocate at Lightbend, we introduce Akka from A to Z, starting with a tour from the humble actor and finishing all the way at the clustered systems level. Specifically, we will review:
*How Akka Actors behave, create systems, and manage supervision and routing
*The way Akka embraces Reactive Streams with Akka Streams and Alpakka
*How various components of the Akka toolkit provide out-of-the-box solutions for distributed data, distributed persistence, pub-sub, and ES/CQRS
*How Akka works with microservices, and brings this functionality into Lagom and Play Frameworks
*Looking at Akka clusters, how Akka is used to build distributed clustered systems incorporate clusters within clusters
*What’s needed to orchestrate and deploy complete Reactive Systems
Introduction to Akka.NET and Akka.Clusterpetabridge
Demands and expectations for .NET developers have never been higher.
We're expected increase customer value against higher and higher expectations; deliver our services and content across a greater variety of devices; retain, analyze, and use ever-growing volumes of data faster; and to do all of this while being available 24/7.
That's a tall order, but it's one that is being done successfully by .NET companies all over the world - right now.
In this webinar, lead by Petabridge CEO Aaron Stannard, we're going to cover how the obvious ways of scaling software in the past are doomed to fail and how .NET shops are developing and deploying their own distributed systems to tackle these problems efficiently and effectively using Akka.NET and Akka.Cluster.
Continuous Deployment with Akka.Cluster and Kubernetes (Akka.NET)petabridge
In this 60 minute long webinar Petabridge and Akka.NET co-founder Aaron Stannard you will learn about how companies ranging from the Fortune 500 to brand new startups are changing the way the build .NET applications to leverage the very latest offerings from Microsoft and the .NET open source community.
You'll learn how and why companies are moving their applications onto .NET Core; rearchitecting them to use Akka.NET for fault tolerance, scalability, and the ability to respond to customers in real-time; containerizing them with Docker; putting everything together using Kubernetes for orchestration on-premise or on the cloud with Azure Container Services.
This session will provide an overview of how all of these technologies fit together and why companies are adopting them.
Technologies Referenced: Akka, Typesafe Reactive Platform
Technical Level: Introductory
Audience: Senior Developers, Architects
Presenter: Konrad Malawski, Akka Software Engineer, Typesafe, Inc.
Akka is a runtime framework for building resilient, distributed applications in Java or Scala. In this webinar, Konrad Malawski discusses the roadmap and features of the upcoming Akka 2.4.0 and reveals three upcoming enhancements that enterprises will receive in the latest certified, tested build of Typesafe Reactive Platform.
Akka Split Brain Resolver (SBR)
Akka SBR provides advanced recovery scenarios in Akka Clusters, improving on the safety of Akka’s automatic resolution to avoid cascading partitioning.
Akka Support for Docker and NAT
Run Akka Clusters in Docker containers or NAT with complete hostname and port visibility on Java 6+ and Akka 2.3.11+
Akka Long-Term Support
Receive Akka 2.4 support for Java 6, Java 7, and Scala 2.10
Akka Revealed: A JVM Architect's Journey From Resilient Actors To Scalable Cl...Lightbend
By now, you’ve probably heard of Akka, the JVM toolkit for building scalable, resilient and resource efficient applications in Java or Scala. With over 12 open-source and commercial modules in the toolkit, Akka takes developers from actors on a single JVM, all the way out to network partition healing and clusters of servers distributed across fleets of JVMs. But with such a broad range of features, how can Architects and Developers grok Akka from a high-level perspective?
In this technical webinar by Hugh McKee, O’Reilly author and Developer Advocate at Lightbend, we introduce Akka from A to Z, starting with a tour from the humble actor and finishing all the way at the clustered systems level. Specifically, we will review:
*How Akka Actors behave, create systems, and manage supervision and routing
*The way Akka embraces Reactive Streams with Akka Streams and Alpakka
*How various components of the Akka toolkit provide out-of-the-box solutions for distributed data, distributed persistence, pub-sub, and ES/CQRS
*How Akka works with microservices, and brings this functionality into Lagom and Play Frameworks
*Looking at Akka clusters, how Akka is used to build distributed clustered systems incorporate clusters within clusters
*What’s needed to orchestrate and deploy complete Reactive Systems
Introduction to Akka.NET and Akka.Clusterpetabridge
Demands and expectations for .NET developers have never been higher.
We're expected increase customer value against higher and higher expectations; deliver our services and content across a greater variety of devices; retain, analyze, and use ever-growing volumes of data faster; and to do all of this while being available 24/7.
That's a tall order, but it's one that is being done successfully by .NET companies all over the world - right now.
In this webinar, lead by Petabridge CEO Aaron Stannard, we're going to cover how the obvious ways of scaling software in the past are doomed to fail and how .NET shops are developing and deploying their own distributed systems to tackle these problems efficiently and effectively using Akka.NET and Akka.Cluster.
Continuous Deployment with Akka.Cluster and Kubernetes (Akka.NET)petabridge
In this 60 minute long webinar Petabridge and Akka.NET co-founder Aaron Stannard you will learn about how companies ranging from the Fortune 500 to brand new startups are changing the way the build .NET applications to leverage the very latest offerings from Microsoft and the .NET open source community.
You'll learn how and why companies are moving their applications onto .NET Core; rearchitecting them to use Akka.NET for fault tolerance, scalability, and the ability to respond to customers in real-time; containerizing them with Docker; putting everything together using Kubernetes for orchestration on-premise or on the cloud with Azure Container Services.
This session will provide an overview of how all of these technologies fit together and why companies are adopting them.
Technologies Referenced: Akka, Typesafe Reactive Platform
Technical Level: Introductory
Audience: Senior Developers, Architects
Presenter: Konrad Malawski, Akka Software Engineer, Typesafe, Inc.
Akka is a runtime framework for building resilient, distributed applications in Java or Scala. In this webinar, Konrad Malawski discusses the roadmap and features of the upcoming Akka 2.4.0 and reveals three upcoming enhancements that enterprises will receive in the latest certified, tested build of Typesafe Reactive Platform.
Akka Split Brain Resolver (SBR)
Akka SBR provides advanced recovery scenarios in Akka Clusters, improving on the safety of Akka’s automatic resolution to avoid cascading partitioning.
Akka Support for Docker and NAT
Run Akka Clusters in Docker containers or NAT with complete hostname and port visibility on Java 6+ and Akka 2.3.11+
Akka Long-Term Support
Receive Akka 2.4 support for Java 6, Java 7, and Scala 2.10
Application development has come a long way. From client-server, to desktop, to web based applications served by monolithic application servers, the need to serve billions of users and hundreds of devices have become crucial to today's business. Typesafe Reactive Platform helps you to modernize your applications by transforming the most critical parts into microservice-style architectures which support extremely high workloads and allow you to serve millions of end-users.
Understanding Akka Streams, Back Pressure, and Asynchronous ArchitecturesLightbend
The term 'streams' has been getting pretty overloaded recently–it's hard to know where to best use different technologies with streams in the name. In this talk by noted hAkker Konrad Malawski, we'll disambiguate what streams are and what they aren't, taking a deeper look into Akka Streams (the implementation) and Reactive Streams (the standard).
You'll be introduced to a number of real life scenarios where applying back-pressure helps to keep your systems fast and healthy at the same time. While the focus is mainly on the Akka Streams implementation, the general principles apply to any kind of asynchronous, message-driven architectures.
DotNext 2020 - When and How to Use the Actor Model and Akka.NETpetabridge
The actor model is an old computer science concept, originating in 1973 and it laid dormant is largely a thought experiment for most of its history until the rise of the Internet. Now in the era of cheap, commodity cloud computing the actor model is staging a major comeback across all programming languages and runtimes, both for building distributed systems and for creating reactive mobile or desktop applications.
In this talk, we will introduce the actor model through the use of Akka.NET, the most popular distributed actor model framework in .NET. We'll talk about what sorts of problems it solves well when you should use it, and what are some of the adoption costs and overhead involved in using a tool like Akka.NET.
By the time you're finished with this talk, you should be familiar with most of the major Akka.NET and actor model concepts, basic Akka.NET syntax, and some ideas for how you might be able to use actors in your place of work. This talk is intended for developers, architects, and team leads.
Go Reactive: Event-Driven, Scalable, Resilient & Responsive SystemsJonas Bonér
The demands and expectations for applications have changed dramatically in recent years. Applications today are deployed on a wide range of infrastructure; from mobile devices up to thousands of nodes running in the cloud—all powered by multi-core processors. They need to be rich and collaborative, have a real-time feel with millisecond response time and should never stop running. Additionally, modern applications are a mashup of external services that need to be consumed and composed to provide the features at hand.
We are seeing a new type of applications emerging to address these new challenges—these are being called Reactive Applications. In this talk we will discuss four key traits of Reactive; Event-Driven, Scalable, Resilient and Responsive—how they impact application design, how they interact, their supporting technologies and techniques, how to think when designing and building them—all to make it easier for you and your team to Go Reactive.
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...Lightbend
Akka Streams and its amazing handling of streaming with back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially in use cases where the amount of work continues to increase as you’re processing it. This is where back-pressure really shines.
In this talk for Architects and Dev Managers by Akara Sucharitakul, Principal MTS for Global Platform Frameworks at PayPal, Inc., we look at how back-pressure based on Akka Streams and Kafka is being used at PayPal to handle very bursty workloads.
In addition, Akara will also share experiences in creating a platform based on Akka and Akka Streams that currently processes over 1 billion transactions per day (on just 8 VMs), with the aim of helping teams adopt these technologies. In this webinar, you will:
*Start with a sample web crawler use case to examine what happens when each processing pass expands to a larger and larger workload to process.
*Review how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
*Look at lessons learned, plus some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
A Tale of Two APIs: Using Spark Streaming In ProductionLightbend
Fast Data architectures are the answer to the increasing need for the enterprise to process and analyze continuous streams of data to accelerate decision making and become reactive to the particular characteristics of their market.
Apache Spark is a popular framework for data analytics. Its capabilities include SQL-based analytics, dataflow processing, graph analytics and a rich library of built-in machine learning algorithms. These libraries can be combined to address a wide range of requirements for large-scale data analytics.
To address Fast Data flows, Spark offers two API's: The mature Spark Streaming and its younger sibling, Structured Streaming. In this talk, we are going to introduce both APIs. Using practical examples, you will get a taste of each one and obtain guidance on how to choose the right one for your application.
Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...Lightbend
Microservices. Streaming data. Event Sourcing and CQRS. Concurrency, routing, self-healing, persistence, clustering… You get the picture. The Akka toolkit makes all of this simple for Java and Scala developers at Amazon, LinkedIn, Starbucks, Verizon and others. So how does Akka provide all these features out of the box?
Join Hugh McKee, Akka expert and Developer Advocate at Lightbend, on an illustrated journey that goes deep into how Akka works–from individual Akka actors to fully distributed clusters across multiple datacenters.
Resilient Applications with Akka Persistence - Scaladays 2014Björn Antonsson
In this presentation you will learn how to leverage the features introduced in Akka Persistence: opt-in at-least-once delivery semantics between actors and the ability to recover application state after a crash. Both are implemented by storing immutable facts in a persisted append-only log. We will show you how to create persistent actors using command and event sourcing, replicate events with reliable communication, scale out and improve resilience with clustering.
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...Lightbend
Things were easier when all our data used to be offline, analyzed overnight in batches. Now our data is online, in motion, and generated constantly. For architects, developers and their businesses, this means that there is an urgent need for tools and applications that can deliver real-time (or near real-time) streaming ETL capabilities.
In this session by Konrad Malawski, author, speaker and Senior Akka Engineer at Lightbend, you will learn how to build these streaming ETL pipelines with Akka Streams, Alpakka and Apache Kafka, and why they matter to enterprises that are increasingly turning to streaming Fast Data applications.
Akka is a runtime framework for building resilient, distributed applications in Java or Scala. In this webinar, Konrad Malawski discusses the roadmap and features of the upcoming Akka 2.4.0 and reveals three upcoming enhancements that enterprises will receive in the latest certified, tested build of Typesafe Reactive Platform.
Akka Split Brain Resolver (SBR)
Akka SBR provides advanced recovery scenarios in Akka Clusters, improving on the safety of Akka’s automatic resolution to avoid cascading partitioning.
Akka Support for Docker and NAT
Run Akka Clusters in Docker containers or NAT with complete hostname and port visibility on Java 6+ and Akka 2.3.11+
Akka Long-Term Support
Receive Akka 2.4 support for Java 6, Java 7, and Scala 2.10
Introduction to Akka 2. Explains what Akka's actors are all about and how to utilize them to write scalable and fault-tolerant systems.
Talk given at JavaZone 2012.
Modernizing Infrastructures for Fast Data with Spark, Kafka, Cassandra, React...Lightbend
The Big Data industry emerged in response to the unprecedented sizes of data sets collected by Internet companies and the particular needs they had to store and use that data.
Today, the need to process that data more quickly is morphing Big Data architectures into Fast Data architectures. This session discusses the forces driving this trend and the most popular tools that have emerged to address particular design challenges:
Spark - For sophisticated processing of data streams, as well as traditional batch-mode processing.
Kafka - For durable and scalable ingestion and distribution of data streams.
Cassandra - For scalable, flexible persistence.
Reactive Platform: Lagom, Akka, and Play - For integration of other components and building microservices.
Mesos - For cluster resource management.
---
About the presenter:
Dean Wampler, Ph.D. is the Architect for Big Data Products and Services and a member of the office of the CTO at Lightbend. He is designing the product strategy and technical architecture for Lightbend's Spark on Mesos products and emerging streaming tools built around Spark and Lightbend’s ConductR and Akka products. Dean has written books on Scala, Functional Programming, and Hive for O'Reilly. He speaks at and co-organizes many industry conferences. He also organizes several Chicago-area user groups and contributes to many open-source projects, including Apache Spark. Dean has a Ph.D. in Physics from the University of Washington.
Topic Modeling via Tensor Factorization - Use Case for Apache REEFSergiy Matusevych
Slides from my talk at 2015 Hadoop Summit.
Topic modeling – a task of discovering hidden thematic structure in a set of documents – is an important problem of modern machine learning. Despite great progress in recent years, scaling it for the large number of topics and massive corpora remains a challenge. We share our experiences in building a high performance distributed system for topic modeling using Apache REEF framework. We start with introduction into topic modeling via Latent Dirichlet Allocation, and describe our new LDA inference algorithm based on tensor factorization. Finally, we demonstrate how using Apache REEF framework helped us to implement the system in a few lines of clean readable code.
Proper distribution of functionalities throughout many machines is very hard, especially when we leave those decisions for later. Akka toolkit gives us many tools for scaling out and we can start using them very early in a development process, enhancing our chances of success. In this introductory talk, I want to go through a very simple example and show snippets of single-noded and sharded implementations.
Application development has come a long way. From client-server, to desktop, to web based applications served by monolithic application servers, the need to serve billions of users and hundreds of devices have become crucial to today's business. Typesafe Reactive Platform helps you to modernize your applications by transforming the most critical parts into microservice-style architectures which support extremely high workloads and allow you to serve millions of end-users.
Understanding Akka Streams, Back Pressure, and Asynchronous ArchitecturesLightbend
The term 'streams' has been getting pretty overloaded recently–it's hard to know where to best use different technologies with streams in the name. In this talk by noted hAkker Konrad Malawski, we'll disambiguate what streams are and what they aren't, taking a deeper look into Akka Streams (the implementation) and Reactive Streams (the standard).
You'll be introduced to a number of real life scenarios where applying back-pressure helps to keep your systems fast and healthy at the same time. While the focus is mainly on the Akka Streams implementation, the general principles apply to any kind of asynchronous, message-driven architectures.
DotNext 2020 - When and How to Use the Actor Model and Akka.NETpetabridge
The actor model is an old computer science concept, originating in 1973 and it laid dormant is largely a thought experiment for most of its history until the rise of the Internet. Now in the era of cheap, commodity cloud computing the actor model is staging a major comeback across all programming languages and runtimes, both for building distributed systems and for creating reactive mobile or desktop applications.
In this talk, we will introduce the actor model through the use of Akka.NET, the most popular distributed actor model framework in .NET. We'll talk about what sorts of problems it solves well when you should use it, and what are some of the adoption costs and overhead involved in using a tool like Akka.NET.
By the time you're finished with this talk, you should be familiar with most of the major Akka.NET and actor model concepts, basic Akka.NET syntax, and some ideas for how you might be able to use actors in your place of work. This talk is intended for developers, architects, and team leads.
Go Reactive: Event-Driven, Scalable, Resilient & Responsive SystemsJonas Bonér
The demands and expectations for applications have changed dramatically in recent years. Applications today are deployed on a wide range of infrastructure; from mobile devices up to thousands of nodes running in the cloud—all powered by multi-core processors. They need to be rich and collaborative, have a real-time feel with millisecond response time and should never stop running. Additionally, modern applications are a mashup of external services that need to be consumed and composed to provide the features at hand.
We are seeing a new type of applications emerging to address these new challenges—these are being called Reactive Applications. In this talk we will discuss four key traits of Reactive; Event-Driven, Scalable, Resilient and Responsive—how they impact application design, how they interact, their supporting technologies and techniques, how to think when designing and building them—all to make it easier for you and your team to Go Reactive.
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...Lightbend
Akka Streams and its amazing handling of streaming with back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially in use cases where the amount of work continues to increase as you’re processing it. This is where back-pressure really shines.
In this talk for Architects and Dev Managers by Akara Sucharitakul, Principal MTS for Global Platform Frameworks at PayPal, Inc., we look at how back-pressure based on Akka Streams and Kafka is being used at PayPal to handle very bursty workloads.
In addition, Akara will also share experiences in creating a platform based on Akka and Akka Streams that currently processes over 1 billion transactions per day (on just 8 VMs), with the aim of helping teams adopt these technologies. In this webinar, you will:
*Start with a sample web crawler use case to examine what happens when each processing pass expands to a larger and larger workload to process.
*Review how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
*Look at lessons learned, plus some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
A Tale of Two APIs: Using Spark Streaming In ProductionLightbend
Fast Data architectures are the answer to the increasing need for the enterprise to process and analyze continuous streams of data to accelerate decision making and become reactive to the particular characteristics of their market.
Apache Spark is a popular framework for data analytics. Its capabilities include SQL-based analytics, dataflow processing, graph analytics and a rich library of built-in machine learning algorithms. These libraries can be combined to address a wide range of requirements for large-scale data analytics.
To address Fast Data flows, Spark offers two API's: The mature Spark Streaming and its younger sibling, Structured Streaming. In this talk, we are going to introduce both APIs. Using practical examples, you will get a taste of each one and obtain guidance on how to choose the right one for your application.
Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...Lightbend
Microservices. Streaming data. Event Sourcing and CQRS. Concurrency, routing, self-healing, persistence, clustering… You get the picture. The Akka toolkit makes all of this simple for Java and Scala developers at Amazon, LinkedIn, Starbucks, Verizon and others. So how does Akka provide all these features out of the box?
Join Hugh McKee, Akka expert and Developer Advocate at Lightbend, on an illustrated journey that goes deep into how Akka works–from individual Akka actors to fully distributed clusters across multiple datacenters.
Resilient Applications with Akka Persistence - Scaladays 2014Björn Antonsson
In this presentation you will learn how to leverage the features introduced in Akka Persistence: opt-in at-least-once delivery semantics between actors and the ability to recover application state after a crash. Both are implemented by storing immutable facts in a persisted append-only log. We will show you how to create persistent actors using command and event sourcing, replicate events with reliable communication, scale out and improve resilience with clustering.
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...Lightbend
Things were easier when all our data used to be offline, analyzed overnight in batches. Now our data is online, in motion, and generated constantly. For architects, developers and their businesses, this means that there is an urgent need for tools and applications that can deliver real-time (or near real-time) streaming ETL capabilities.
In this session by Konrad Malawski, author, speaker and Senior Akka Engineer at Lightbend, you will learn how to build these streaming ETL pipelines with Akka Streams, Alpakka and Apache Kafka, and why they matter to enterprises that are increasingly turning to streaming Fast Data applications.
Akka is a runtime framework for building resilient, distributed applications in Java or Scala. In this webinar, Konrad Malawski discusses the roadmap and features of the upcoming Akka 2.4.0 and reveals three upcoming enhancements that enterprises will receive in the latest certified, tested build of Typesafe Reactive Platform.
Akka Split Brain Resolver (SBR)
Akka SBR provides advanced recovery scenarios in Akka Clusters, improving on the safety of Akka’s automatic resolution to avoid cascading partitioning.
Akka Support for Docker and NAT
Run Akka Clusters in Docker containers or NAT with complete hostname and port visibility on Java 6+ and Akka 2.3.11+
Akka Long-Term Support
Receive Akka 2.4 support for Java 6, Java 7, and Scala 2.10
Introduction to Akka 2. Explains what Akka's actors are all about and how to utilize them to write scalable and fault-tolerant systems.
Talk given at JavaZone 2012.
Modernizing Infrastructures for Fast Data with Spark, Kafka, Cassandra, React...Lightbend
The Big Data industry emerged in response to the unprecedented sizes of data sets collected by Internet companies and the particular needs they had to store and use that data.
Today, the need to process that data more quickly is morphing Big Data architectures into Fast Data architectures. This session discusses the forces driving this trend and the most popular tools that have emerged to address particular design challenges:
Spark - For sophisticated processing of data streams, as well as traditional batch-mode processing.
Kafka - For durable and scalable ingestion and distribution of data streams.
Cassandra - For scalable, flexible persistence.
Reactive Platform: Lagom, Akka, and Play - For integration of other components and building microservices.
Mesos - For cluster resource management.
---
About the presenter:
Dean Wampler, Ph.D. is the Architect for Big Data Products and Services and a member of the office of the CTO at Lightbend. He is designing the product strategy and technical architecture for Lightbend's Spark on Mesos products and emerging streaming tools built around Spark and Lightbend’s ConductR and Akka products. Dean has written books on Scala, Functional Programming, and Hive for O'Reilly. He speaks at and co-organizes many industry conferences. He also organizes several Chicago-area user groups and contributes to many open-source projects, including Apache Spark. Dean has a Ph.D. in Physics from the University of Washington.
Topic Modeling via Tensor Factorization - Use Case for Apache REEFSergiy Matusevych
Slides from my talk at 2015 Hadoop Summit.
Topic modeling – a task of discovering hidden thematic structure in a set of documents – is an important problem of modern machine learning. Despite great progress in recent years, scaling it for the large number of topics and massive corpora remains a challenge. We share our experiences in building a high performance distributed system for topic modeling using Apache REEF framework. We start with introduction into topic modeling via Latent Dirichlet Allocation, and describe our new LDA inference algorithm based on tensor factorization. Finally, we demonstrate how using Apache REEF framework helped us to implement the system in a few lines of clean readable code.
Proper distribution of functionalities throughout many machines is very hard, especially when we leave those decisions for later. Akka toolkit gives us many tools for scaling out and we can start using them very early in a development process, enhancing our chances of success. In this introductory talk, I want to go through a very simple example and show snippets of single-noded and sharded implementations.
This is a brief introduction to Akka.Cluster:
- what is Akka.Cluster?
- what does it do for me / why should I care?
- when do I use it?
- how do I use it?
This slide shows you how to use Akka cluster in Java.
Source Code: https://github.com/jiayun/akka_samples
If you want to use the links in slide, please download the pdf file.
Slides for my talk event-sourced architectures with Akka. Discusses Akka Persistence as mechanism to do event-sourcing. Presented at Javaone 2014 and Jfokus 2015.
This was a talk from DDDSW on Akka.Net, the actor model, concurrency and reactive. It covered what they are as well as an example use case and the lessons learned when running in that use case.
CQRS (Command Query Responsibility Segregation) was all the hype in .NET architecture circles a few years back. But has it faded away? Is it old news? I argue that it hasn't, and the concepts of CQRS are alive and well and probably more widely accepted and considered today than a few years ago. From event-driven systems to the Reactive Manifesto, the principles of CQRS are with us and impacting many different tools. In this session, we'll explore those CQRS principles and how they have manifested themselves in the architectures of choice today. You'll come away with a greater appreciation of CQRS and ideas on how to incorporate those principles in your applications today.
오픈 소스 Actor Framework 인 Akka.NET 을 통해 온라인 게임 서버를 어떻게 구현할 수 있는지를 설명합니다. Actor Model 에 대한 기본 이해부터 Scale-out 가능한 게임 서버 구축까지 전반적인 내용에 대해 알 수 있습니다. 설명을 위해 클라이언트는 Unity3D 를 사용할 예정입니다.
Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...Jonas Bonér
Akka is the platform for the next generation event-driven, scalable and fault-tolerant architectures on the JVM
We believe that writing correct concurrent, fault-tolerant and scalable applications is too hard. Most of the time it's because we are using the wrong tools and the wrong level of abstraction.
Akka is here to change that.
Using the Actor Model together with Software Transactional Memory we raise the abstraction level and provides a better platform to build correct concurrent and scalable applications.
For fault-tolerance we adopt the "Let it crash" / "Embrace failure" model which have been used with great success in the telecom industry to build applications that self-heals, systems that never stop.
Actors also provides the abstraction for transparent distribution and the basis for truly scalable and fault-tolerant applications.
Akka is Open Source and available under the Apache 2 License.
Reactive applications are becoming a de-facto industry standard and, if employed correctly, toolkits like Lightbend Reactive Platform make the implementation easier than ever. But design of these systems might be challenging as it requires particular mindset shift to tackle problems we might not be used to.
In this talk, we’re going to discuss the most common things I’ve seen in the field that prevented applications working as expected. I’d like to talk about typical pitfalls that might cause problems, about trade-offs that might not be fully understood and important choices that might be overlooked. These include persistent actors pitfalls, tackling of network partitions, proper implementations of graceful shutdown or distributed transactions, trade-offs of micro-services or actors and more.
This talk should be interesting for anyone who is thinking about, implementing, or has already deployed a reactive application. My goal is to provide a comprehensive explanation of common problems to be sure they won’t be repeated by fellow developers. The talk is a little bit more focused on the Lightbend platform but understanding of the concepts we are going to talk about should be beneficial for everyone interested in this field.
Beyond the Query – Bringing Complex Access Patterns to NoSQL with DataStax - ...StampedeCon
Learn how to model beyond traditional direct access in Apache Cassandra. Utilizing the DataStax platform to harness the power of Spark and Solr to perform search, analytics, and complex operations in place on your Cassandra data!
Beyond the Query: A Cassandra + Solr + Spark Love Triangle Using Datastax Ent...DataStax Academy
Wait! Back away from the Cassandra 2ndary index. It’s ok for some use cases, but it’s not an easy button. "But I need to search through a bunch of columns to look for the data and I want to do some regression analysis… and I can’t model that in C*, even after watching all of Patrick McFadins videos. What do I do?” The answer, dear developer, is in DSE Search and Analytics. With it’s easy Solr API and Spark integration so you can search and analyze data stored in your Cassandra database until your heart’s content. Take our hand. WE will show you how.
A Cassandra + Solr + Spark Love Triangle Using DataStax EnterprisePatrick McFadin
Wait! Back away from the Cassandra 2ndary index. It’s ok for some use cases, but it’s not an easy button. "But I need to search through a bunch of columns to look for the data and I want to do some regression analysis… and I can’t model that in C*, even after watching all of Patrick McFadins videos. What do I do?” The answer, dear developer, is in DSE Search and Analytics. With it’s easy Solr API and Spark integration so you can search and analyze data stored in your Cassandra database until your heart’s content. Take our hand. WE will show you how.
Solr Exchange: Introduction to SolrCloudthelabdude
SolrCloud is a set of features in Apache Solr that enable elastic scaling of search indexes using sharding and replication. In this presentation, Tim Potter will provide an architectural overview of SolrCloud and highlight its most important features. Specifically, Tim covers topics such as: sharding, replication, ZooKeeper fundamentals, leaders/replicas, and failure/recovery scenarios. Any discussion of a complex distributed system would not be complete without a discussion of the CAP theorem. Mr. Potter will describe why Solr is considered a CP system and how that impacts the design of a search application.
Apache Cassandra operations have the reputation to be simple on single datacenter deployments and / or low volume clusters but they become way more complex on high latency multi-datacenter clusters with high volume and / or high throughout: basic Apache Cassandra operations such as repairs, compactions or hints delivery can have dramatic consequences even on a healthy high latency multi-datacenter cluster.
In this presentation, Julien will go through Apache Cassandra mutli-datacenter concepts first then show multi-datacenter operations essentials in details: bootstrapping new nodes and / or datacenter, repairs strategy, Java GC tuning, OS tuning, Apache Cassandra configuration and monitoring.
Based on his 3 years experience managing a multi-datacenter cluster against Apache Cassandra 2.0, 2.1, 2.2 and 3.0, Julien will give you tips on how to anticipate and prevent / mitigate issues related to basic Apache Cassandra operations with a multi-datacenter cluster.
Apache Cassandra operations have the reputation to be simple on single datacenter deployments and / or low volume clusters but they become way more complex on high latency multi-datacenter clusters with high volume and / or high throughout: basic Apache Cassandra operations such as repairs, compactions or hints delivery can have dramatic consequences even on a healthy high latency multi-datacenter cluster.
In this presentation, Julien will go through Apache Cassandra mutli-datacenter concepts first then show multi-datacenter operations essentials in details: bootstrapping new nodes and / or datacenter, repairs strategy, Java GC tuning, OS tuning, Apache Cassandra configuration and monitoring.
Based on his 3 years experience managing a multi-datacenter cluster against Apache Cassandra 2.0, 2.1, 2.2 and 3.0, Julien will give you tips on how to anticipate and prevent / mitigate issues related to basic Apache Cassandra operations with a multi-datacenter cluster.
About the Speaker
Julien Anguenot VP Software Engineering, iland Internet Solutions, Corp
Julien currently serves as iland's Vice President of Software Engineering. Prior to joining iland, Mr. Anguenot held tech leadership positions at several open source content management vendors and tech startups in Europe and in the U.S. Julien is a long time Open Source software advocate, contributor and speaker: Zope, ZODB, Nuxeo contributor, Zope and OpenStack foundations member, his talks includes Apache Con, Cassandra summit, OpenStack summit, The WWW Conference or still EuroPython.
Scaling Through Partitioning and Shard Splitting in Solr 4thelabdude
Over the past several months, Solr has reached a critical milestone of being able to elastically scale-out to handle indexes reaching into the hundreds of millions of documents. At Dachis Group, we've scaled our largest Solr 4 index to nearly 900M documents and growing. As our index grows, so does our need to manage this growth.
In practice, it's common for indexes to continue to grow as organizations acquire new data. Over time, even the best designed Solr cluster will reach a point where individual shards are too large to maintain query performance. In this Webinar, you'll learn about new features in Solr to help manage large-scale clusters. Specifically, we'll cover data partitioning and shard splitting.
Partitioning helps you organize subsets of data based on data contained in your documents, such as a date or customer ID. We'll see how to use custom hashing to route documents to specific shards during indexing. Shard splitting allows you to split a large shard into 2 smaller shards to increase parallelism during query execution.
Attendees will come away from this presentation with a real-world use case that proves Solr 4 is elastically scalable, stable, and is production ready.
Apache Cassandra operations have the reputation to be quite simple against single datacenter clusters and / or low volume clusters but they become way more complex against high latency multi-datacenter clusters: basic operations such as repair, compaction or hints delivery can have dramatic consequences even on a healthy cluster.
In this presentation, Julien will go through Cassandra operations in details: bootstrapping new nodes and / or datacenter, repair strategies, compaction strategies, GC tuning, OS tuning, large batch of data removal and Apache Cassandra upgrade strategy.
Julien will give you tips and techniques on how to anticipate issues inherent to multi-datacenter cluster: how and what to monitor, hardware and network considerations as well as data model and application level bad design / anti-patterns that can affect your multi-datacenter cluster performances.
Introduction to Real Application Cluster
RAC - Savior of DBA
Oracle Clusterware (Platform on Platform)
RAC Startup sequence
RAC Architecture
RAC Components
Single Instance on RAC
Node Eviction
Important Log directories in RAC.
Tips to monitor and improve the RAC environment.
2014 OSDC Talk: Introduction to Percona XtraDB Cluster and HAProxyBo-Yi Wu
Introduction to Percona XtraDB Cluster and HAProxy in 2014 OSDC talk.
Blog: http://blog.wu-boy.com/2014/04/osdc-2014-talk-introduction-to-percona-xtradb-cluster-and-haproxy/
OSDC: http://osdc.tw/program/2014-day2-10.html#content
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
5. Actors
„The actor model in computer science is a mathematical model of concurrent computation
that treats "actors" as the universal primitives of concurrent computation. In response to a
message that it receives, an actor can:
• make local decisions,
• create more actors,
• send more messages, and
• determine how to respond to the next message received.
Actors may modify private state, but can only affect each other through messages (avoiding
the need for any locks).“
https://en.wikipedia.org/wiki/Actor_model
8. Failure Detector
• The Phi Accrual Failure Detector phi = -log10(1 - F(timeSinceLastHeartbeat))
• Each node monitored by small number of other nodes determined using Hash Ring
• Output: confidence that a node is unreachable
• Also notices when it becomes reachable again
9. Akka Cluster
• Cluster Membership managed using Gossip Protocol
• Dynamo based system
• Subscribe to cluster state events
• Roles
• Restrictions on number of nodes possible (also per role)
• When Gossip Convergence is reached, a leader can deterministically be determined
• Head of list of nodes in alphanumeric order
• Leader joins / removes members
• Leader can auto-down members
• Join manually or to Seed Nodes
• The first seed node joins itself
11. case class Gossip(
members: immutable.SortedSet[Member], // sorted set of members with their status, sorted by address
overview: GossipOverview = GossipOverview(),
version: VectorClock = VectorClock()) // vector clock version
case class GossipOverview(
seen: Set[UniqueAddress] = Set.empty,
reachability: Reachability = Reachability.empty)
case class VectorClock(
versions: TreeMap[VectorClock.Node, Long] = TreeMap.empty[VectorClock.Node, Long]) {
/**
* Compare two vector clocks. The outcome will be one of the following:
* <p/>
* {{{
* 1. Clock 1 is SAME (==) as Clock 2 iff for all i c1(i) == c2(i)
* 2. Clock 1 is BEFORE (<) Clock 2 iff for all i c1(i) <= c2(i)
* and there exist a j such that c1(j) < c2(j)
* 3. Clock 1 is AFTER (>) Clock 2 iff for all i c1(i) >= c2(i)
* and there exist a j such that c1(j) > c2(j).
* 4. Clock 1 is CONCURRENT (<>) to Clock 2 otherwise.
* }}}
*/
def compareTo(that: VectorClock): Ordering = {
compareOnlyTo(that, FullOrder)
}
}
13. Cluster Singletons
• e.g. single point of entry, centralized routing logic, …
• live on the oldest node
• ClusterSingletonManager started on each node
• ClusterSingletonProxy for accessing current Singleton
16. Cluster Singletons
• e.g. single point of entry, centralized routing logic, …
• live on the oldest node
• ClusterSingletonManager started on each node
• ClusterSingletonProxy for accessing current Singleton
• caveats:
• Single point of bottleneck
• Must recover state on migration
• In case of split brain, multiple singletons
17. Distributed PubSub
• DistributedPubSubMediator started on all nodes
• Subscriptions are gossiped, eventually consistent
• modes Publish, Group Publish, Send
• used e.g. for cluster wide config, chat system, …
18. Cluster Sharding
• Distribute Work
• Workload partitioned by shard key derived from message
• Messages must be serializable
• Each node is responsible for n shards and each shard is allocated to one node
• ShardRegion is entry point for messages and controls workers
• ShardCoordinator singleton assigns shards
• Shards distributed by no of workers by default
• Shards migrate for rebalancing or on failure
• Shard assignments can be persisted
• Running Workers per shard can be remembered.
• workers must step down
• Workers must persist state if they need it after migration
42. Live Example...
• config
• scaling horizontally
• cluster client
• multi JVM test
• singleton monitoring throughput and lifecycle
43. Caveats and Lessons learned
• Remoting Setup
• TLS certificates rolling update
• difficult to test if new settings work whole the old ones are still there
• In our case: export restricted crypto
• Not too critical, if noticed early in rolling upgrade
• Adjust Failure detector settings to your environment
• quite strict for us
• must accept higher latencies & short interruptions in cloud environment
• Configure internal & external hostname in containerized / NATed / … environment
• hostname and IP are completely different for Akka!
44. Caveats and Lessons learned
• Cluster Setup
• Currently rather static hardware environment
• Joining to list of seed nodes.
• Had a split brain once, need to carefully restart the right part of the cluster
• Log cluster state each node sees (or use JMX)!
• preventing split brain
• maybe disable auto down
• split brain resolver
• adjust failure detector settings to your environment
• restart order youngest to oldest to minimize singleton migrations
45. Caveats and Lessons learned
• Sharding
• Recovery of persistent actors should be planned (might take time)
• If shard coordinator fails to recover you're doomed
• separate journal for internal sharding state
• can be cleaned if cluster is shutdown
• ICE delete journal
• Will allow for ShardCoordinator recovery
• Will make shard allocation state in ShardRegions inconsistent
• Fix state by rolling restart
46. Shutdown
// Play can not stop accepting requests.
// Fail healthcheck, so the loadbalancer removes this node.
GlobalHealthcheck.fail(reason = shuttingDown)
// migrate all shards to other nodes, stop accepting new ones
val cluster = Cluster(context.system)
context.watch(region)
region ! ShardRegion.GracefulShutdown
// After shutdown of ShardRegion, shutdown the ActorSystem
case Terminated(`region`) =>
cluster.leave(cluster.selfAddress)
cluster.registerOnMemberRemoved {
system.terminate
}
// After shutdown of the ActorSystem, shutdown the App
system.registerOnTermination {
System.exit(0)
}
