This document outlines the agenda for Season 2 Episode 2 of an event. The episode will include:
1. Lightning talks on various Reactive and Rx topics from different speakers
2. Guest speakers Jake Wharton from Square, Matt Ingenthron from Couchbase, and Will Sargent from Typesafe
3. More details on RxJava and composable reactive functions
We designed a new framework, made for Microservices. Making it easier for developers to build microservices-based systems – systems that communicate asynchronously, self-heal, scale elastically and remain responsive no matter what bad stuff is happening.
And all this without the pain of selecting and mixing components, from a plethora of libraries that were originally built for other things.
In this presentation, we reveal this new way for Java developers to not only understand and begin building microservices, but also to seamlessly push them into staging and production
Lightbend Lagom: Microservices Just Rightmircodotta
Microservices architecture are becoming a de-facto industry standard, but are you satisfied with the current state of the art? We are not, as we believe that building microservices today is more challenging than it should be. Lagom is here to take on this challenge. First, Lagom is opinionated and it will take some of the hard decisions for you, guiding you to produce microservices that adheres to the Reactive tenents. Second, Lagom was built from the ground up around you, the developer, to push your productivity to the next level. If you are familiar with the Play Framework's development environment, imagine that but tuned for building microservices; we are sure you are going to love it! Third, Lagom comes with batteries included for deploying in production: going from development to production could not be easier. In this session, you will get an introduction to the Lightbend Lagom framework. There will be code and live demos to show you in practice how it works and what you can do with it, making you fully equipped to build your next microservices with Lightbend Lagom!
NATS: Simple, Secure and Scalable Messaging For the Cloud Native Erawallyqs
The majority of middleware and messaging systems in use were built in a time that did not have the concept of scale and real-time data that developers operate in today.
With the rise of Cloud Native and Microservices architectures as a design principle and the emphasis on simplicity, speed, and flexibility that come with it, developers need a messaging protocol to match.
Enter NATS. NATS is a remarkably lightweight messaging protocol, and extremely flexible and resilient. It is just a few MB in size, and can scale to publish tens of millions of message from a single server.
We designed a new framework, made for Microservices. Making it easier for developers to build microservices-based systems – systems that communicate asynchronously, self-heal, scale elastically and remain responsive no matter what bad stuff is happening.
And all this without the pain of selecting and mixing components, from a plethora of libraries that were originally built for other things.
In this presentation, we reveal this new way for Java developers to not only understand and begin building microservices, but also to seamlessly push them into staging and production
Lightbend Lagom: Microservices Just Rightmircodotta
Microservices architecture are becoming a de-facto industry standard, but are you satisfied with the current state of the art? We are not, as we believe that building microservices today is more challenging than it should be. Lagom is here to take on this challenge. First, Lagom is opinionated and it will take some of the hard decisions for you, guiding you to produce microservices that adheres to the Reactive tenents. Second, Lagom was built from the ground up around you, the developer, to push your productivity to the next level. If you are familiar with the Play Framework's development environment, imagine that but tuned for building microservices; we are sure you are going to love it! Third, Lagom comes with batteries included for deploying in production: going from development to production could not be easier. In this session, you will get an introduction to the Lightbend Lagom framework. There will be code and live demos to show you in practice how it works and what you can do with it, making you fully equipped to build your next microservices with Lightbend Lagom!
NATS: Simple, Secure and Scalable Messaging For the Cloud Native Erawallyqs
The majority of middleware and messaging systems in use were built in a time that did not have the concept of scale and real-time data that developers operate in today.
With the rise of Cloud Native and Microservices architectures as a design principle and the emphasis on simplicity, speed, and flexibility that come with it, developers need a messaging protocol to match.
Enter NATS. NATS is a remarkably lightweight messaging protocol, and extremely flexible and resilient. It is just a few MB in size, and can scale to publish tens of millions of message from a single server.
Java Day 2021, WeAreDevelopers, 2021-09-01, online: Moritz Kammerer (@Moritz Kammerer, Expert Software Engineer at QAware).
== Please download slides in case they are blurred! ===
In this talk, we took a look at how Microservices can be developed with Micronaut. Have a look if it has kept its promises.
Monitoring, the Prometheus Way - Julius Voltz, Prometheus Docker, Inc.
Prometheus is an opinionated metrics collection and monitoring system that is particularly well suited to accommodate modern workloads like containers and micro-services. To achieve these goals, it radically breaks away from existing systems and follows very different design principles. In this talk, Prometheus founder Julius Volz will explain these design principles and how they apply to dockerized applications. This will provide insight useful to newcomers wanting to start on the right foot in the land of container monitoring, but also to veterans wanting to quickly map their existing knowledge to Prometheus concepts. In particular, a demo will show Prometheus in action together with a Docker Swarm cluster.
Netflix Open Source Meetup Season 4 Episode 2aspyker
In this episode, we will take a close look at 2 different approaches to high-throughput/low-latency data stores, developed by Netflix.
The first, EVCache, is a battle-tested distributed memcached-backed data store, optimized for the cloud. You will also hear about the road ahead for EVCache it evolves into an L1/L2 cache over RAM and SSDs.
The second, Dynomite, is a framework to make any non-distributed data-store, distributed. Netflix's first implementation of Dynomite is based on Redis.
Come learn about the products' features and hear from Thomson and Reuters, Diego Pacheco from Ilegra and other third party speakers, internal and external to Netflix, on how these products fit in their stack and roadmap.
S3, Cassandra or Outer Space? Dumping Time Series Data using Spark - Demi Be...Codemotion
Vast volume of our processed data is Time Series data and once you start working with distributed systems, you start tackling many scale and performance problems: How to handle missing data?Should I handle both serving and backed process or separating them out? Best Performance for Money? In the talk we will tell the tale of all of the transformations we’ve made to our data model@Windward, some of the problems we’ve handled, review the multiple data persistency layers like: S3, MongoDB, Apache Cassandra, MySQL. And I’ll try my best NOT to answer the question “Which one of them is the Best?"
QCon NYC: Distributed systems in practice, in theoryAysylu Greenberg
Modern systems in production rely on decades of computer science research. Over time, new architectural patterns emerge that enable more resilient and robust systems. In this talk, we'll discuss some of these patterns from systems I've worked on at Google and the related work that provide insights into the motivations behind them.
Now that you have your apps running on K8s, wondering how to get the response time that you need ? Tuning applications to get the performance that you need can be challenging. When you have to tune a number of microservices in Kubernetes to fix a response time or a throughput issue, it can get really overwhelming. This talk looks at some common performance issues and ways to solve them and more importantly the tools that can help you.
We will also be specifically looking at Kruize that helps to not only right size your containers but also optimize the runtimes.
( https://github.com/kruize/kruize )
Container orchestration: the cold war - Giulio De Donato - Codemotion Rome 2017Codemotion
L’ecosistema degli orchestratori di container è in rapido movimento, una galassia di piattaforme e framework. Come si fa a scegliere quello giusto per le vostre esigenze? Vediamo tutti gli orchestratori in commercio, con i loro pro e contro: DC/OS, Kubernetes, Docker e anche quelli meno famosi ma saranno promesse, e anche le dinamiche e le scelte fatte.
Kubernetes advanced sheduling
- Taint and tolerant
- Affinity (Node & inter pod)
Learn how to place Pod like (same or different) node, rack, zone, region
(BDT318) How Netflix Handles Up To 8 Million Events Per SecondAmazon Web Services
In this session, Netflix provides an overview of Keystone, their new data pipeline. The session covers how Netflix migrated from Suro to Keystone, including the reasons behind the transition and the challenges of zero loss while processing over 400 billion events daily. The session covers in detail how they deploy, operate, and scale Kafka, Samza, Docker, and Apache Mesos in AWS to manage 8 million events & 17 GB per second during peak.
Slides from a presentation by Monal Daxini at Disney, Glendale CA about Netflix Open Source Software, Cloud Data Persistence, and Cassandra best Practices
"[WORKSHOP] K8S for developers", Denis RomanukFwdays
"It seems that perfection is attained, not when there is nothing more to add, but when there is nothing more to take away." - Antoine de Saint Exupéry
We can talk a lot about Kubernetes. But does the whole info is really need for just begining? Let's take away everything, except the really needed, and show a way for ones, who still uses only docker.
Let's consider:
Why do developers need Kubernetes and what does it looks like?
An attempt to automate ops, or why does it was a reinvention of Kubernetes
What does kube consists of?
How to get a Kubernetes?
From docker-compose to c
What is a Helm and why it's difficult without one?
What is "local development on Kubernetes"?
CRD & Operators
Netflix Open Source Meetup Season 4 Episode 1aspyker
Learn more about how we are evolving our open source. In our evolution we’ll discuss how we are approaching project lifecycles, metrics we are tracking that give us insight into the health of our key projects, and how we are working to make this clear to the communities involved with our projects.
Also, we will discuss one of most recent key open source releases – Spinnaker (http://spinnaker.io/). Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence. Spinnaker powers thousands of deployments per day across the Netflix service.
NetflixOSS Meetup S3 E1, covering latest components in Distributed Databases, Telemetry systems, Big Data tools and more. Speakers from Netflix, IBM Watson, Pivotal and Nike Digital
Java Day 2021, WeAreDevelopers, 2021-09-01, online: Moritz Kammerer (@Moritz Kammerer, Expert Software Engineer at QAware).
== Please download slides in case they are blurred! ===
In this talk, we took a look at how Microservices can be developed with Micronaut. Have a look if it has kept its promises.
Monitoring, the Prometheus Way - Julius Voltz, Prometheus Docker, Inc.
Prometheus is an opinionated metrics collection and monitoring system that is particularly well suited to accommodate modern workloads like containers and micro-services. To achieve these goals, it radically breaks away from existing systems and follows very different design principles. In this talk, Prometheus founder Julius Volz will explain these design principles and how they apply to dockerized applications. This will provide insight useful to newcomers wanting to start on the right foot in the land of container monitoring, but also to veterans wanting to quickly map their existing knowledge to Prometheus concepts. In particular, a demo will show Prometheus in action together with a Docker Swarm cluster.
Netflix Open Source Meetup Season 4 Episode 2aspyker
In this episode, we will take a close look at 2 different approaches to high-throughput/low-latency data stores, developed by Netflix.
The first, EVCache, is a battle-tested distributed memcached-backed data store, optimized for the cloud. You will also hear about the road ahead for EVCache it evolves into an L1/L2 cache over RAM and SSDs.
The second, Dynomite, is a framework to make any non-distributed data-store, distributed. Netflix's first implementation of Dynomite is based on Redis.
Come learn about the products' features and hear from Thomson and Reuters, Diego Pacheco from Ilegra and other third party speakers, internal and external to Netflix, on how these products fit in their stack and roadmap.
S3, Cassandra or Outer Space? Dumping Time Series Data using Spark - Demi Be...Codemotion
Vast volume of our processed data is Time Series data and once you start working with distributed systems, you start tackling many scale and performance problems: How to handle missing data?Should I handle both serving and backed process or separating them out? Best Performance for Money? In the talk we will tell the tale of all of the transformations we’ve made to our data model@Windward, some of the problems we’ve handled, review the multiple data persistency layers like: S3, MongoDB, Apache Cassandra, MySQL. And I’ll try my best NOT to answer the question “Which one of them is the Best?"
QCon NYC: Distributed systems in practice, in theoryAysylu Greenberg
Modern systems in production rely on decades of computer science research. Over time, new architectural patterns emerge that enable more resilient and robust systems. In this talk, we'll discuss some of these patterns from systems I've worked on at Google and the related work that provide insights into the motivations behind them.
Now that you have your apps running on K8s, wondering how to get the response time that you need ? Tuning applications to get the performance that you need can be challenging. When you have to tune a number of microservices in Kubernetes to fix a response time or a throughput issue, it can get really overwhelming. This talk looks at some common performance issues and ways to solve them and more importantly the tools that can help you.
We will also be specifically looking at Kruize that helps to not only right size your containers but also optimize the runtimes.
( https://github.com/kruize/kruize )
Container orchestration: the cold war - Giulio De Donato - Codemotion Rome 2017Codemotion
L’ecosistema degli orchestratori di container è in rapido movimento, una galassia di piattaforme e framework. Come si fa a scegliere quello giusto per le vostre esigenze? Vediamo tutti gli orchestratori in commercio, con i loro pro e contro: DC/OS, Kubernetes, Docker e anche quelli meno famosi ma saranno promesse, e anche le dinamiche e le scelte fatte.
Kubernetes advanced sheduling
- Taint and tolerant
- Affinity (Node & inter pod)
Learn how to place Pod like (same or different) node, rack, zone, region
(BDT318) How Netflix Handles Up To 8 Million Events Per SecondAmazon Web Services
In this session, Netflix provides an overview of Keystone, their new data pipeline. The session covers how Netflix migrated from Suro to Keystone, including the reasons behind the transition and the challenges of zero loss while processing over 400 billion events daily. The session covers in detail how they deploy, operate, and scale Kafka, Samza, Docker, and Apache Mesos in AWS to manage 8 million events & 17 GB per second during peak.
Slides from a presentation by Monal Daxini at Disney, Glendale CA about Netflix Open Source Software, Cloud Data Persistence, and Cassandra best Practices
"[WORKSHOP] K8S for developers", Denis RomanukFwdays
"It seems that perfection is attained, not when there is nothing more to add, but when there is nothing more to take away." - Antoine de Saint Exupéry
We can talk a lot about Kubernetes. But does the whole info is really need for just begining? Let's take away everything, except the really needed, and show a way for ones, who still uses only docker.
Let's consider:
Why do developers need Kubernetes and what does it looks like?
An attempt to automate ops, or why does it was a reinvention of Kubernetes
What does kube consists of?
How to get a Kubernetes?
From docker-compose to c
What is a Helm and why it's difficult without one?
What is "local development on Kubernetes"?
CRD & Operators
Netflix Open Source Meetup Season 4 Episode 1aspyker
Learn more about how we are evolving our open source. In our evolution we’ll discuss how we are approaching project lifecycles, metrics we are tracking that give us insight into the health of our key projects, and how we are working to make this clear to the communities involved with our projects.
Also, we will discuss one of most recent key open source releases – Spinnaker (http://spinnaker.io/). Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence. Spinnaker powers thousands of deployments per day across the Netflix service.
NetflixOSS Meetup S3 E1, covering latest components in Distributed Databases, Telemetry systems, Big Data tools and more. Speakers from Netflix, IBM Watson, Pivotal and Nike Digital
Example of using Kotlin lang features for writing DSL for Spark-Cassandra connector. Comparison Kotlin lang DSL features with similar features in others JVM languages (Scala, Groovy).
서버 개발자가 바라 본 Functional Reactive Programming with RxJava - SpringCamp2015NAVER / MusicPlatform
youtube : https://youtu.be/E_Bgv9upahI
비동기 이벤트 기반의 라이브러리로만 생각 했던 RxJava가 지금 이 시대 프로그래머에게 닥쳐 올 커다란 메시지라는 사실을 알게 된 지금. 현장에서 직접 느낀 RxJava의 본질인 Function Reactive Programming(FRP)에 대해 우리가 잘 아는 Java 이야기로 풀어 보고 ReactiveX(RxJava) 개발을 위한 서버 환경에 대한 이해와 SpringFramework, Netty에서의 RxJava를 어떻게 이용 하고 개발 했는지 공유 하고자 합니다.
Part presentation, part debate about the future of the language while touching base on the current state of the industry with respect to ES6/ES2015, and the possibilities of using it today in web applications and frameworks, the different options, and the things to keep in mind. Additionally, we will do a walk-through on the new features included in ES7/ES2016 draft, and those that are being discussed for ES8/ES2017.
A practical guide to using RxJava on Android. Tips for improving your app architecture with reactive programming. What are the advantages and disadvantages of using RxJava over standard architecture? And how to connect with other popular Android libraries?
Presented at Droidcon Greece 2016.
[JEEConf-2017] RxJava as a key component in mature Big Data productIgor Lozynskyi
Zoomdata is the fastest visual analytics product for big data available on the market. Here we will talk about how RxJava became the key component in its architecture. I want to share our experience with reactive programming and RxJava so that audience will learn about “good” and “bad” stuff based on a real and cool product. Besides, we will talk about performance implications and integration with “big names”, including Apache Spark.
Testing Android apps based on Dagger and RxJava Droidcon UKFabio Collini
Writing reliable tests is not easy for many reasons, especially when asynchronous code is involved. Some libraries can be used to create a testable architecture (for example Dagger and RxJava) and to simplify testing (Mockito, AssertJ and DaggerMock). In this talk you'll learn how to write JVM and Espresso tests with particular attention on how to replace real objects with mocks and how to test asynchronous RxJava code. In this talk, you will also explore:
how to take advantage of Mockito and other libraries to write good JVM and Espresso tests in both Java and Kotlin
how to use DaggerMock (an open source library available on github to avoid boilerplate code in tests
how to test asynchronous RxJava code using a JVM test
Building Scalable Stateless Applications with RxJavaRick Warren
RxJava is a lightweight open-source library, originally from Netflix, that makes it easy to compose asynchronous data sources and operations. This presentation is a high-level intro to this library and how it can fit into your application.
Similar to NetflixOSS season 2 episode 2 - Reactive / Async (20)
Arc305 how netflix leverages multiple regions to increase availability an i...Ruslan Meshenberg
Learn how to make your services more resilient and available by embracing principles of isolation and redundancy. See details of 2 projects - Isthmus and Active/Active to learn how Netflix architects for availability in multi-regional environment.
Netflix
has
built
and
deployed
a
scalable
global
Platorm
as
a
Service.
Key
components
of
the
Netflix
PaaS
are
being
released
as
Open
Source
projects
so
you
can
build
your
own
custom
PaaS
HEAP SORT ILLUSTRATED WITH HEAPIFY, BUILD HEAP FOR DYNAMIC ARRAYS.
Heap sort is a comparison-based sorting technique based on Binary Heap data structure. It is similar to the selection sort where we first find the minimum element and place the minimum element at the beginning. Repeat the same process for the remaining elements.
Harnessing WebAssembly for Real-time Stateless Streaming PipelinesChristina Lin
Traditionally, dealing with real-time data pipelines has involved significant overhead, even for straightforward tasks like data transformation or masking. However, in this talk, we’ll venture into the dynamic realm of WebAssembly (WASM) and discover how it can revolutionize the creation of stateless streaming pipelines within a Kafka (Redpanda) broker. These pipelines are adept at managing low-latency, high-data-volume scenarios.
6th International Conference on Machine Learning & Applications (CMLA 2024)ClaraZara1
6th International Conference on Machine Learning & Applications (CMLA 2024) will provide an excellent international forum for sharing knowledge and results in theory, methodology and applications of on Machine Learning & Applications.
We have compiled the most important slides from each speaker's presentation. This year’s compilation, available for free, captures the key insights and contributions shared during the DfMAy 2024 conference.
KuberTENes Birthday Bash Guadalajara - K8sGPT first impressionsVictor Morales
K8sGPT is a tool that analyzes and diagnoses Kubernetes clusters. This presentation was used to share the requirements and dependencies to deploy K8sGPT in a local environment.
Hierarchical Digital Twin of a Naval Power SystemKerry Sado
A hierarchical digital twin of a Naval DC power system has been developed and experimentally verified. Similar to other state-of-the-art digital twins, this technology creates a digital replica of the physical system executed in real-time or faster, which can modify hardware controls. However, its advantage stems from distributing computational efforts by utilizing a hierarchical structure composed of lower-level digital twin blocks and a higher-level system digital twin. Each digital twin block is associated with a physical subsystem of the hardware and communicates with a singular system digital twin, which creates a system-level response. By extracting information from each level of the hierarchy, power system controls of the hardware were reconfigured autonomously. This hierarchical digital twin development offers several advantages over other digital twins, particularly in the field of naval power systems. The hierarchical structure allows for greater computational efficiency and scalability while the ability to autonomously reconfigure hardware controls offers increased flexibility and responsiveness. The hierarchical decomposition and models utilized were well aligned with the physical twin, as indicated by the maximum deviations between the developed digital twin hierarchy and the hardware.
31. Tuning for an acceptable latency
distribution
client: ab
server CPU
usage varies
with load
32. RxNetty vs Tomcat
Event loop model more efficient than thread pools:
● Reduces thread CPU overheads
● Reduced thread lock contention
● Better work scheduling: platform chooses, instead of the
kernel scheduling threads
● Warmer CPU caches and memory locality on NUMA:
event worker threads can have better CPU affinity
33. RxNetty Micro Benchmarking
Server: AWS c3.xlarge, tuned to reduce perturbations
Client: Multi-threaded (wrk -t 4 -c 100/400), 1 min warmup
● Sufficient load to exhaust CPU capacity.
Active Benchmarking Methodology:
● Tools: sysstat, perf_events, SystemTap, flame graphs, …
● USE Method, static performance tuning, etc.
● Target analyzed during benchmark confirm target results
and identify (and tune) true limiters.
35. Hello World
● All platforms are <100us for Hello World, so their CPU
overheads are already negligible for this workload.
○ Consider application compute times of >1ms
● Hence testing “Hello Netflix” as well: a basic Netflix
service Hello World with dependency requests.
● Although, Hello World max latency (not pictured) was
still much higher for Tomcat...
36. Visualizing Tomcat Overheads
● Flame graph of
Java CPU time
for Hello World
benchmark
● Thread and I/O
management
overheads
37. Visualizing RxNetty Overheads
● Flame graph of Java CPU
time for Hello World
● Time in read() and write()
(doing I/O; ie, doing “work”)
dominates
● Flame graphs also studied
of kernel and user-time
● Many improvements found
and fixed
38. Blog post forthcoming… Keywords:
● Oracle JDK 1.8.0 (others tried); Xmx & Xms; different
GCs and settings, eg -XX:+UseConcMarkSweepGC
● Apache-tomcat 7.0.54: tested BIO, NIO, APR;
maxThreads=150 or 512; maxQueueSize=up to 400;
acceptCount=up to 100; disabled access log, ...
● Node.js 0.6.12 with clustering (faster than 0.10/0.11)
● vert.x 2.1 with -instances 4
● Java profilers: Google lightweight java profiler; JFR; …
● perf_events; SystemTap; flame graphs, ...
40. Netflix IPC Stack (1.0)
A
p
a
c
h
e
H
T
T
P
C
l
i
e
n
t
Eureka (Service Registry)
Server (Karyon)
Apache
Tomcat
Client
H
y
s
t
r
i
x
E
V
C
a
c
h
e
Ribbon
Load
Balancing
Eureka
Integration
Metrics
(Servo)
Bootstrapping (Governator)
Metrics (Servo)
Admin ConsoleHTTP
Eureka Integration
Registration
Fetch Registry
A Blocking Architecture
41. Netflix IPC Stack (2.0)
Client (Ribbon 2.0)
Eureka (Service Registry)
Server (Karyon)
Ribbon Transport
Load
Balancing
Eureka
Integration
Metrics
(Servo)
Bootstrapping (Governator)
Metrics (Servo)
Admin Console
HTTP
Eureka Integration
Registration
Fetch Registry
Ribbon
Hystrix
EVCache
R
x
N
e
t
t
y
RxNetty
UDP
TCP
WebSockets
SSE
A Completely Reactive
Architecture
42. “We want an extremely performant IPC library”
Heard this before?
51. Karyon 2.0
● Adds Netflix constructs (governator, eureka, archaius,
etc.) on RxNetty.
● Provides extensions (servlet, jersey, etc.) to RxNetty.
● More palatable to application developers.
52. Example
@ArchaiusBootstrap
@KaryonBootstrap(name = "hello-netflix-oss")
@Modules(include = {HelloNossApp.KaryonJerseyModuleImpl.class, KaryonWebAdminModule.class, KaryonEurekaModule.class})
public final class HelloNossApp {
public static class KaryonJerseyModuleImpl extends KaryonJerseyModule {
protected void configure() {
super.configure();
bind(AuthenticationService.class).to(AuthenticationServiceImpl.class);
}
public int serverPort() { return 8888; }
public int shutdownPort() { return 8899; }
public void configureInterceptors(GovernatorHttpInterceptorSupport<ByteBuf, ByteBuf> interceptorSupport) {
interceptorSupport.forUri("/hello").interceptIn(AuthInterceptor.class);
}
}
}
54. ● Tendency to become “fat” with boilerplate
code
● Hystrix integration
● Go “async”
● “Declare” a client, rather than “write” a client
Ribbon 2.0 - Motivation
56. Ribbon 2.0 - New “ribbon” module
● Template support for building HTTP request
● Rx style non-blocking and blocking APIs
● Hystrix built-in for fault tolerance
● Pluggable cache access with EVCache
implementation
● Annotation based client creation as an
alternative
58. Ribbon 2: Template Example
HttpResourceGroup movieService = Ribbon.createHttpResourceGroup("movieService");
HttpRequestTemplate<ByteBuf> template1 =
movieService.newRequestTemplate("recommendationsByUserId", ByteBuf.class)
.withMethod("GET")
.withUriTemplate("/users/{userId}/recommendations")
.withFallbackProvider(new RecommendationServiceFallbackHandler());
RibbonRequest<ByteBuf> request = template1.requestBuilder()
.withRequestProperty("userId", “user1”).build();
Observable<ByteBuf> result = request.toObservable(); // non blocking
ByteBuf result = request.execute(); // blocking
59. Ribbon 2.0 - Other New Features
● Ribbon’s async module - “ribbon-transport”
○ HTTP, TCP and UDP clients on top of RxNetty with
load balancing capability
● New load balancing features
○ Command pattern load balancing APIs implemented
with Rx for easy integration for third party client
○ Shuffle sharding server list for fault isolation
91. Motivations with Couchbase Java
➢ Simplify ...
➢ Even *more* performance
➢ Build for the JVM, not for the Java language
○ Have Scala, JRuby already being worked on
92. History of Concurrency
Core Java component, spymemcached, has
used Futures since 2006 or so.
➢ Always been ahead of most other database
like clients, however…
➢ Difficult for developers to code against
○ Dumb benchmarks turn into FAQs
96. What we’re Building On
➢ A set of new components
○ RxJava (both internal and public interface)
○ Netty (internal only)
○ LMAX Disruptor
97. RxJava for Couchbase
➢ Will be part of the client’s public API
○ Helps us support Java 8, but is also backward
compatible
■ Generally get a great Java 8 experience for free!
➢ Also using RxJava extensively internally
○ Breaking the project into core and language façade
helps build toward Scala and others
98. Getting Referenced Elements
client.asyncGet("posts::1").addListener(future -> {
if (future.isDone() && !future.isCancelled()) {
String content = (String) future.get();
List<String> ids =
comments_ids_from_json(content);
int i = 0;
for (String id : ids) {
if (i++ == 5) {
break;
}
client.asyncGet(id).addListener(future1 -> {
if (future1.isDone() && !future1.isCancelled()) {
System.out.println(future1.get());
}
});
}
}
});
99. Getting Referenced Elements Rx Way
bucket
.get("posts::1")
.map(document -> document.content().getArray("comments"))
.flatMap(comments -> Observable.from(comments.toList()))
.take(5)
.filter(o -> o instanceof String)
.flatMap(o -> bucket.get((String) o))
.subscribe(doc -> System.out.println(doc.content()));
107. Akka with Futures
Akka to Future:
val myFuture = actorRef ? message
Future to Akka:
futureStuff pipeTo actorRef
108. Assuming a Services based API
val fooFuture = fooService.get(1)
val barFuture = barService.get(2)
val quuxFuture = quuxService.get(3)
...Simple, right?
109. Not so fast
val fooFuture = fooService.get(1)
val barFuture = barService.get(foo.barId)
val quuxFuture = quuxService.get(bar.quuxId)
...wat.
110. FLATMAP THAT
fooService.get(1).map { maybeFoo =>
maybeFoo.flatMap { foo =>
val barFuture = barService.get(foo.barId)
// do the same for bar & quux
}
}
111. For comprehensions!
for {
maybeFoo <- fooService.get(1)
} yield {
for { // Option / Future does not compose
foo <- maybeFoo
} yield ...
}
112. Scala Async!
async {
val maybeFoo = await(fooService.get(1))
maybeFoo.flatMap { foo =>
val bar = await(barService.get(2))
}
}
113. Scala Async Sadness
COMPILE ERROR.
“await must not be used inside a closure nested
within an async block” == no flatMap of that.
114. Monads don’t compose
But before reaching for readers/transformers:
● Use small methods
● Short circuit
● Use loan patterns