SlideShare a Scribd company logo

Crossroads of Asynchrony and Graceful Degradation

•
•
C4Media
C4Media

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1VmbI3t. Nitesh Kant describes how embracing asynchrony in the Netflix applications, from networking to business processing, creates gracefully degrading and highly resilient applications. Filmed at qconsf.com. Nitesh Kant is an engineer in Netflix’s Edge Gateway team, working on Netflix’s asynchronous Inter Process Communication stack. He is the author of RxNetty which forms the core of this stack and is currently moving Zuul to this new architecture.

Technology
1 of 159
Crossroads of asynchrony and graceful degradation Nitesh Kant,! Software Engineer, ! Netflix Edge Engineering.! ! @NiteshKant
InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /netflix-asynchronous-apps
Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
Nitesh Kant Who Am I? ❖ Engineer, Edge Engineering, Netflix.! ❖ Core contributor, RxNetty*! ❖ Contributor, Zuul** * https://github.com/ReactiveX/RxNetty ** https://github.com/Netflix/zuul @NiteshKant
How do systems fail?
A simple example. Showing a movie on Netflix.
Video Metadata
Video Bookmark
Video Rating
! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! Disclaimer: This is an example and not an exact representation of the processing
Synchronicity ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! Disclaimer: This is an example and not an exact representation of the processing
The bigger picture Price of being synchronous? ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! Disclaimer: This is an example and not an exact representation of the processing
In a microservices world Edge Service Ratings ServiceVideo Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
In a microservices world Edge Service Server threadpool Thread Thread Thread Thread Thread getMovieMetadata(movieId) Disclaimer: This is an example and not an exact representation of the processing
In a microservices world Edge Service Server threadpool Thread Thread Thread Thread Thread getMovieMetadata(movieId) getBookmark(movieId, userId) Disclaimer: This is an example and not an exact representation of the processing
In a microservices world Edge Service Server threadpool Thread Thread Thread Thread Thread getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) Disclaimer: This is an example and not an exact representation of the processing
Busy thread time = Sum of the time taken to make all 3 service calls
How do systems fail? 1. Latency Latency is your worst enemy in a synchronous world.
Edge Service Server threadpool Thread Thread Thread Thread Thread getRatings(movieId) Disclaimer: This is an example and not an exact representation of the processing
Disclaimer: This is an example and not an exact representation of the processing Ratings Service Edge Service Server threadpool Thread Thread Thread Thread Thread getRatings(movieId)
Disclaimer: This is an example and not an exact representation of the processing Edge Service getRatings(movieId) Server threadpool Thread Thread Thread Thread Thread
Edge Service Disclaimer: This is an example and not an exact representation of the processing getRatings(movieId) Server threadpool Thread Thread Thread Thread Thread
Edge Service Server threadpool Thread Thread Thread Thread Thread Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread getRatings(movieId)
Edge Service Server threadpool Thread Thread Thread Thread Thread Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread getRatings(movieId)
Edge Service Server threadpool Thread Thread Thread Thread Thread Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread getRatings(movieId)
Managing client thread pools Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread
Managing client thread pools
Managing client thread pools
Managing client thread pools
Managing client thread pools
Clients have become our babies
Clients have become our babies Edge Service Server threadpool Thread Thread Thread Thread Thread getMovieMetadata(movieId) Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread Client Threadpool Thread Thread Thread Thread Thread Client Threadpool Thread Thread Thread Thread Thread getBookmark(movieId, userId) getRatings(movieId)
Clients have become our babies
Untuned/Wrongly tuned clients cause many outages.
Have we exchanged a bigger problem with a smaller one?
How do systems fail? 2. Overload Abusive clients, recovery spikes, special events ….
We did a load test… https://github.com/Netflix-Skunkworks/ WSPerfLab Hello Netflix!
Detailed analysis available online: https://github.com/Netflix-Skunkworks/WSPerfLab/blob/master/test-results/RxNetty_vs_Tomcat_April2015.pdf
! Graceful This isn’t graceful degradation!
This happens at high CPU usage.
This happens at high CPU usage. So, don’t let the system reach that limit…
This happens at high CPU usage. So, don’t let the system reach that limit… a.k.a Throttling.
Fairness? One abusive request type can penalize other request paths.
How do systems fail? 3. Thundering herds The failure after recovery….
Retries Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing
Retries Edge Service Disclaimer: This is an example and not an exact representation of the processing Video Metadata Service Cluster
Retries Edge Service Disclaimer: This is an example and not an exact representation of the processing Video Metadata Service Cluster
Retries Edge Service Disclaimer: This is an example and not an exact representation of the processing Video Metadata Service Cluster
Retries are useful in steady state…. …but…
Retries Edge Service Disclaimer: This is an example and not an exact representation of the processing Video Metadata Service Cluster
Our systems are missing empathy.
Because they lack knowledge about the peers.
Knowledge comes from various signals..
Ability to adapt to those signals is important.
This can not adapt… ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! Disclaimer: This is an example and not an exact representation of the processing
Asynchrony It is the key to success.
What should be async?
What should be async? Edge Service Video Metadata Service
What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) Application logic
What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) I/O I/O I/O Application logic I/O
What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) I/O I/O I/O Application logic I/O Network protocol
Key aspects of being async.
Key aspects of being async. 1. Lifecycle control
Lifecycle control Start processing Stop processing
Key aspects of being async. 2. Flow control
Flow control When How much
Key aspects of being async. 3. Function composition
Function composition ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }!
Function composition Composing the processing ! of a method into a single control point. public Observable<Movie> getMovie(String movieId) {! return Observable.zip(getMovieMetadata(movieId),! getBookmark(movieId, userId),! getRatings(movieId),! (meta,bmark,rating)->new Movie(meta,bmark,rating));! }!
Composing the processing ! of a method into a single control point. Flow & Lifecycle Control with
What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) I/O I/O I/O Application logic I/O Network protocol
I/O Edge Service Server threadpool Thread Thread Thread Thread Thread Client Threadpool Thread Thread Thread Thread Thread getRatings(movieId)
I/O Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloops = f (Number of cores)
I/O Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection Connections multiplexed ! on a ! single eventloop.
I/O Disclaimer: This is an example and not an exact representation of the processing Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection
I/O Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Clients share the eventloops! with! the server.
I/O Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection All clients share ! the! same eventloop
Composing the processing ! of a service into a single control point. Flow & Lifecycle Control with
What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) I/O I/O I/O Application logic I/O Network protocol
Network Protocol HTTP/1.1?
HTTP/1.1 GET /movie?id=1 HTTP/1.1
HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1
HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1
HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK! ID: 1! …
HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK! ID: 1! … HTTP/1.1 200 OK! ID: 2! …
HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK! ID: 1! … HTTP/1.1 200 OK! ID: 2! … HTTP/1.1 200 OK! ID: 3! …
HTTP/1.1 GET /movie?id=3 GET /movie?id=2 GET /movie?id=1 HTTP/1.1 200 ID: 1! … HTTP/1.1 200 ID: 2! … HTTP/1.1 200 ID: 3! … Head Of Line Blocking => Synchronous
Network Protocol HTTP/1.1?
Network Protocol We need a multiplexed bi-directional protocol
Multiplexed GET /movie?id=1 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=3 HTTP/1.1
Bi-directional GET /movie?id=1 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=3 HTTP/1.1 CANCEL
Composing the processing ! of the entire application into a single control point. Flow & Lifecycle Control with
Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing
Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing Observable<Movie>
Observable<Movie> Composing the processing ! of the entire application into a single control point.
Revisiting the failure modes
Latency Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection
Latency Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Impact is localized to the connection.
Latency Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Impact is localized to the connection. An outstanding request has little cost.
An outstanding request has little cost. GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK! … } Any stored state between! request - response! is costly.
Outstanding requests have low cost so Latency is a lesser evil in asynchronous systems.
Overload & Thundering Herds Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Disclaimer: This is an example and not an exact representation of the processing Reduce work done ! when overloaded
Overload & Thundering Herds Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Disclaimer: This is an example and not an exact representation of the processing Reduce work done ! when overloaded Stop accepting! new requests.
Stop accepting new requests Non-blocking I/O gives better control
Stop accepting new requests But … we are still “throttling”
Stop accepting new requests Are we being empathetic?
Request-leasing http://reactivesocket.io/
Request-leasing Peer 1Peer 2 Network connection
Request-leasing Peer 1Peer 2 “Lease” 5 requests for 1 minute. Network connection
Request-leasing Peer 1Peer 2 “Lease” 5 requests for 1 minute. GET /movie?id=1 HTTP/1.1 GET /movie?id=2 HTTP/1.1 Network connection
Server Client 1 Client 2 Client 8
ServerCapacity: 100 RPM Client 1 Client 2 Client 8 “Lease” 10 requests for 1 minute.
ServerCapacity: 100 RPM Client 1 Client 2 Client 8 “Lease” 10 requests for 1 minute. “Lease” 10 requests for 1 minute. “Lease” 10 requests for 1 minute.
ServerCapacity: 100 RPM Client 1 Client 2 Client 8 “Lease” 10 requests for 1 minute. “Lease” 10 requests for 1 minute. “Lease” 10 requests for 1 minute. Reserve Capacity: ! 20 RPM
Time bound lease. “Lease” 10 requests for 1 minute.
Time bound lease. No extra work for cancelling leases. “Lease” 10 requests for 1 minute.
Time bound lease. No extra work for cancelling leases. Receiver controls the flow of requests “Lease” 10 requests for 1 minute.
When things go south
ServerCapacity: 20 RPM Client 1 Client 2 Client 8 “Lease” 5 requests for 1 minute. “Lease” 2 requests for 1 minute. X No more “Lease”
ServerCapacity: 20 RPM Client 1 Client 2 Client 8 “Lease” 5 requests for 1 minute. “Lease” 2 requests for 1 minute. X No more “Lease” Prioritization
Managing client configs?
Threadpools? Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection I/O! is! non-blocking.
Threadpools? Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Application code ! is! non-blocking.
Threadpools? Disclaimer: This is an example and not an exact representation of the processing Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection No blocking/Waiting => Only CPU work
Threadpools? Disclaimer: This is an example and not an exact representation of the processing Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection No blocking/Waiting => Only CPU work So, Eventloops = # of cores
Tuning parameters?
Tuning parameters? X
Tuning parameters? X X
Tuning parameters? X X? ?
Case for timeouts?
Case for timeouts? Read Timeouts Thread Timeouts ✤ Useful in unblocking threads 
 on socket reads. ✤ Business level SLA. ✤ Unblock the calling thread.
Case for timeouts? Read Timeouts Thread Timeouts ✤ Useful in unblocking threads 
 on socket reads. ✤ Business level SLA. ✤ Unblock the calling thread. X X As there are no blocking calls.X
Case for timeouts? Read Timeouts Thread Timeouts ✤ Useful in unblocking threads 
 on socket reads. ✤ Business level SLA. ✤ Unblock the calling thread.
Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing
Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store
Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store
Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store Thread timeouts are pretty invasive at every level
Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store Thread timeouts are pretty invasive at every level Do we need them at every step?
Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing
Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Business timeouts are ! for ! a client request. Disclaimer: This is an example and not an exact representation of the processing
Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing
Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 X Disclaimer: This is an example and not an exact representation of the processing
Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 X X X X X Disclaimer: This is an example and not an exact representation of the processing
Tuning parameters? X X X X
Less tuning
Edge Service Video Metadata Service Rating service C* store C* store Disclaimer: This is an example and not an exact representation of the processing Request Leases Cancellations Observable<Movie>
Edge Service Video Metadata Service Rating service C* store C* store Disclaimer: This is an example and not an exact representation of the processing Request Leases Cancellations Observable<Movie>
! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! public Observable<Movie> getMovie(String movieId) {! return Observable.zip(getMovieMetadata(movieId),! getBookmark(movieId, userId),! getRatings(movieId),! (meta,bmark,rating)->new Movie(meta,bmark,rating));! }!
Resources Asynchronous Function composition : I/O : Network Protocol : https://github.com/ReactiveX/RxJava https://github.com/ReactiveX/RxNetty http://reactivesocket.io/
Nitesh Kant, Engineer, Netflix Edge Gateway @NiteshKant
Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/netflix- asynchronous-apps

Recommended

From Developer to Production, Promoting your Webservices
From Developer to Production, Promoting your WebservicesFrom Developer to Production, Promoting your Webservices
From Developer to Production, Promoting your Webserviceskingsfleet
 
CDI 2.0 is coming
CDI 2.0 is comingCDI 2.0 is coming
CDI 2.0 is comingAntoine Sabot-Durand
 
2015 JavaOne EJB/CDI Alignment
2015 JavaOne EJB/CDI Alignment2015 JavaOne EJB/CDI Alignment
2015 JavaOne EJB/CDI AlignmentDavid Blevins
 
EJB and CDI - Alignment and Strategy
EJB and CDI - Alignment and StrategyEJB and CDI - Alignment and Strategy
EJB and CDI - Alignment and StrategyDavid Delabassee
 
10 Reasons Your Software Sucks - Election 2012 Edition
10 Reasons Your Software Sucks - Election 2012 Edition10 Reasons Your Software Sucks - Election 2012 Edition
10 Reasons Your Software Sucks - Election 2012 EditionCaleb Jenkins
 
zeugnis-zsb
zeugnis-zsbzeugnis-zsb
zeugnis-zsbIvaylo Radev
 
Building a useful target architecture - Myth or reality2
Building a useful target architecture - Myth or reality2Building a useful target architecture - Myth or reality2
Building a useful target architecture - Myth or reality2Regine Deleu
 
Consciousness as a Limitation
Consciousness as a LimitationConsciousness as a Limitation
Consciousness as a LimitationTrieu Nguyen
 

Recommended

From Developer to Production, Promoting your Webservices
From Developer to Production, Promoting your WebservicesFrom Developer to Production, Promoting your Webservices
From Developer to Production, Promoting your Webserviceskingsfleet
 
CDI 2.0 is coming
CDI 2.0 is comingCDI 2.0 is coming
CDI 2.0 is comingAntoine Sabot-Durand
 
2015 JavaOne EJB/CDI Alignment
2015 JavaOne EJB/CDI Alignment2015 JavaOne EJB/CDI Alignment
2015 JavaOne EJB/CDI AlignmentDavid Blevins
 
EJB and CDI - Alignment and Strategy
EJB and CDI - Alignment and StrategyEJB and CDI - Alignment and Strategy
EJB and CDI - Alignment and StrategyDavid Delabassee
 
10 Reasons Your Software Sucks - Election 2012 Edition
10 Reasons Your Software Sucks - Election 2012 Edition10 Reasons Your Software Sucks - Election 2012 Edition
10 Reasons Your Software Sucks - Election 2012 EditionCaleb Jenkins
 
zeugnis-zsb
zeugnis-zsbzeugnis-zsb
zeugnis-zsbIvaylo Radev
 
Building a useful target architecture - Myth or reality2
Building a useful target architecture - Myth or reality2Building a useful target architecture - Myth or reality2
Building a useful target architecture - Myth or reality2Regine Deleu
 
Consciousness as a Limitation
Consciousness as a LimitationConsciousness as a Limitation
Consciousness as a LimitationTrieu Nguyen
 
praktikum_solidarnost_Ivaylo Radev
praktikum_solidarnost_Ivaylo Radevpraktikum_solidarnost_Ivaylo Radev
praktikum_solidarnost_Ivaylo RadevIvaylo Radev
 
Art nouveau &amp; de st ijl
Art nouveau &amp; de st ijlArt nouveau &amp; de st ijl
Art nouveau &amp; de st ijlPawan Singh
 
Does Current Advertising Cause Future Sales?
Does Current Advertising Cause Future Sales?Does Current Advertising Cause Future Sales?
Does Current Advertising Cause Future Sales?Trieu Nguyen
 
Microservices
Microservices Microservices
Microservices Trieu Nguyen
 
Fast Data processing with RFX
Fast Data processing with RFXFast Data processing with RFX
Fast Data processing with RFXTrieu Nguyen
 
Slide 2 collecting, storing and analyzing big data
Slide 2 collecting, storing and analyzing big dataSlide 2 collecting, storing and analyzing big data
Slide 2 collecting, storing and analyzing big dataTrieu Nguyen
 
Application-oriented ping-pong benchmarking: how to assess the real communica...
Application-oriented ping-pong benchmarking: how to assess the real communica...Application-oriented ping-pong benchmarking: how to assess the real communica...
Application-oriented ping-pong benchmarking: how to assess the real communica...Trieu Nguyen
 
Findability Day 2016 - Augmented intelligence
Findability Day 2016 - Augmented intelligenceFindability Day 2016 - Augmented intelligence
Findability Day 2016 - Augmented intelligenceFindwise
 
Reactive Data System in Practice
Reactive Data System in PracticeReactive Data System in Practice
Reactive Data System in PracticeTrieu Nguyen
 
A Day in the Life of a Hadoop Administrator
A Day in the Life of a Hadoop AdministratorA Day in the Life of a Hadoop Administrator
A Day in the Life of a Hadoop AdministratorEdureka!
 
Where is my next jobs in the age of Big Data and Automation
Where is my next jobs in the age of Big Data and AutomationWhere is my next jobs in the age of Big Data and Automation
Where is my next jobs in the age of Big Data and AutomationTrieu Nguyen
 
Introduction and overview ArangoDB query language AQL
Introduction and overview ArangoDB query language AQLIntroduction and overview ArangoDB query language AQL
Introduction and overview ArangoDB query language AQLArangoDB Database
 
Hbase, Cách thức lưu trữ và tìm kiếm
Hbase, Cách thức lưu trữ và tìm kiếmHbase, Cách thức lưu trữ và tìm kiếm
Hbase, Cách thức lưu trữ và tìm kiếmTuan Bach Van
 
2016 Data Science Salary Survey
2016 Data Science Salary Survey2016 Data Science Salary Survey
2016 Data Science Salary SurveyTrieu Nguyen
 
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...Caserta
 
Slide 3 Fast Data processing with kafka, rfx and redis
Slide 3 Fast Data processing with kafka, rfx and redisSlide 3 Fast Data processing with kafka, rfx and redis
Slide 3 Fast Data processing with kafka, rfx and redisTrieu Nguyen
 
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in Production
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in ProductionUpgrade Without the Headache: Best Practices for Upgrading Hadoop in Production
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in ProductionCloudera, Inc.
 
Luan van hadoop-final
Luan van hadoop-finalLuan van hadoop-final
Luan van hadoop-finalnobjta2015
 
Unlocking Business Value Using Data
Unlocking Business Value Using DataUnlocking Business Value Using Data
Unlocking Business Value Using DataSplunk
 
Experience economy
Experience economyExperience economy
Experience economyTrieu Nguyen
 
Architecting Microservices in .Net
Architecting Microservices in .NetArchitecting Microservices in .Net
Architecting Microservices in .NetRichard Banks
 
Big datadc skyfall_preso_v2
Big datadc skyfall_preso_v2Big datadc skyfall_preso_v2
Big datadc skyfall_preso_v2abramsm
 

More Related Content

Viewers also liked

praktikum_solidarnost_Ivaylo Radev
praktikum_solidarnost_Ivaylo Radevpraktikum_solidarnost_Ivaylo Radev
praktikum_solidarnost_Ivaylo RadevIvaylo Radev
 
Art nouveau &amp; de st ijl
Art nouveau &amp; de st ijlArt nouveau &amp; de st ijl
Art nouveau &amp; de st ijlPawan Singh
 
Does Current Advertising Cause Future Sales?
Does Current Advertising Cause Future Sales?Does Current Advertising Cause Future Sales?
Does Current Advertising Cause Future Sales?Trieu Nguyen
 
Fast Data processing with RFX
Fast Data processing with RFXFast Data processing with RFX
Fast Data processing with RFXTrieu Nguyen
 
Slide 2 collecting, storing and analyzing big data
Slide 2 collecting, storing and analyzing big dataSlide 2 collecting, storing and analyzing big data
Slide 2 collecting, storing and analyzing big dataTrieu Nguyen
 
Application-oriented ping-pong benchmarking: how to assess the real communica...
Application-oriented ping-pong benchmarking: how to assess the real communica...Application-oriented ping-pong benchmarking: how to assess the real communica...
Application-oriented ping-pong benchmarking: how to assess the real communica...Trieu Nguyen
 
Findability Day 2016 - Augmented intelligence
Findability Day 2016 - Augmented intelligenceFindability Day 2016 - Augmented intelligence
Findability Day 2016 - Augmented intelligenceFindwise
 
Reactive Data System in Practice
Reactive Data System in PracticeReactive Data System in Practice
Reactive Data System in PracticeTrieu Nguyen
 
A Day in the Life of a Hadoop Administrator
A Day in the Life of a Hadoop AdministratorA Day in the Life of a Hadoop Administrator
A Day in the Life of a Hadoop AdministratorEdureka!
 
Where is my next jobs in the age of Big Data and Automation
Where is my next jobs in the age of Big Data and AutomationWhere is my next jobs in the age of Big Data and Automation
Where is my next jobs in the age of Big Data and AutomationTrieu Nguyen
 
Introduction and overview ArangoDB query language AQL
Introduction and overview ArangoDB query language AQLIntroduction and overview ArangoDB query language AQL
Introduction and overview ArangoDB query language AQLArangoDB Database
 
Hbase, Cách thức lưu trữ và tìm kiếm
Hbase, Cách thức lưu trữ và tìm kiếmHbase, Cách thức lưu trữ và tìm kiếm
Hbase, Cách thức lưu trữ và tìm kiếmTuan Bach Van
 
2016 Data Science Salary Survey
2016 Data Science Salary Survey2016 Data Science Salary Survey
2016 Data Science Salary SurveyTrieu Nguyen
 
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...Caserta
 
Slide 3 Fast Data processing with kafka, rfx and redis
Slide 3 Fast Data processing with kafka, rfx and redisSlide 3 Fast Data processing with kafka, rfx and redis
Slide 3 Fast Data processing with kafka, rfx and redisTrieu Nguyen
 
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in Production
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in ProductionUpgrade Without the Headache: Best Practices for Upgrading Hadoop in Production
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in ProductionCloudera, Inc.
 
Luan van hadoop-final
Luan van hadoop-finalLuan van hadoop-final
Luan van hadoop-finalnobjta2015
 
Unlocking Business Value Using Data
Unlocking Business Value Using DataUnlocking Business Value Using Data
Unlocking Business Value Using DataSplunk
 
Experience economy
Experience economyExperience economy
Experience economyTrieu Nguyen
 

Viewers also liked (20)

praktikum_solidarnost_Ivaylo Radev
praktikum_solidarnost_Ivaylo Radevpraktikum_solidarnost_Ivaylo Radev
praktikum_solidarnost_Ivaylo Radev
 
Art nouveau &amp; de st ijl
Art nouveau &amp; de st ijlArt nouveau &amp; de st ijl
Art nouveau &amp; de st ijl
 
Does Current Advertising Cause Future Sales?
Does Current Advertising Cause Future Sales?Does Current Advertising Cause Future Sales?
Does Current Advertising Cause Future Sales?
 
Microservices
Microservices Microservices
Microservices
 
Fast Data processing with RFX
Fast Data processing with RFXFast Data processing with RFX
Fast Data processing with RFX
 
Slide 2 collecting, storing and analyzing big data
Slide 2 collecting, storing and analyzing big dataSlide 2 collecting, storing and analyzing big data
Slide 2 collecting, storing and analyzing big data
 
Application-oriented ping-pong benchmarking: how to assess the real communica...
Application-oriented ping-pong benchmarking: how to assess the real communica...Application-oriented ping-pong benchmarking: how to assess the real communica...
Application-oriented ping-pong benchmarking: how to assess the real communica...
 
Findability Day 2016 - Augmented intelligence
Findability Day 2016 - Augmented intelligenceFindability Day 2016 - Augmented intelligence
Findability Day 2016 - Augmented intelligence
 
Reactive Data System in Practice
Reactive Data System in PracticeReactive Data System in Practice
Reactive Data System in Practice
 
A Day in the Life of a Hadoop Administrator
A Day in the Life of a Hadoop AdministratorA Day in the Life of a Hadoop Administrator
A Day in the Life of a Hadoop Administrator
 
Where is my next jobs in the age of Big Data and Automation
Where is my next jobs in the age of Big Data and AutomationWhere is my next jobs in the age of Big Data and Automation
Where is my next jobs in the age of Big Data and Automation
 
Introduction and overview ArangoDB query language AQL
Introduction and overview ArangoDB query language AQLIntroduction and overview ArangoDB query language AQL
Introduction and overview ArangoDB query language AQL
 
Hbase, Cách thức lưu trữ và tìm kiếm
Hbase, Cách thức lưu trữ và tìm kiếmHbase, Cách thức lưu trữ và tìm kiếm
Hbase, Cách thức lưu trữ và tìm kiếm
 
2016 Data Science Salary Survey
2016 Data Science Salary Survey2016 Data Science Salary Survey
2016 Data Science Salary Survey
 
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...
 
Slide 3 Fast Data processing with kafka, rfx and redis
Slide 3 Fast Data processing with kafka, rfx and redisSlide 3 Fast Data processing with kafka, rfx and redis
Slide 3 Fast Data processing with kafka, rfx and redis
 
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in Production
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in ProductionUpgrade Without the Headache: Best Practices for Upgrading Hadoop in Production
Upgrade Without the Headache: Best Practices for Upgrading Hadoop in Production
 
Luan van hadoop-final
Luan van hadoop-finalLuan van hadoop-final
Luan van hadoop-final
 
Unlocking Business Value Using Data
Unlocking Business Value Using DataUnlocking Business Value Using Data
Unlocking Business Value Using Data
 
Experience economy
Experience economyExperience economy
Experience economy
 

Similar to Crossroads of Asynchrony and Graceful Degradation

Architecting Microservices in .Net
Architecting Microservices in .NetArchitecting Microservices in .Net
Architecting Microservices in .NetRichard Banks
 
Big datadc skyfall_preso_v2
Big datadc skyfall_preso_v2Big datadc skyfall_preso_v2
Big datadc skyfall_preso_v2abramsm
 
Refacoring vs Rewriting WixStores
Refacoring vs Rewriting WixStoresRefacoring vs Rewriting WixStores
Refacoring vs Rewriting WixStoresDoron Rosenstock
 
Stress Test as a Culture
Stress Test as a CultureStress Test as a Culture
Stress Test as a CultureJoão Moura
 
Netflix: From Zero to Production-Ready in Minutes (QCon 2017)
Netflix: From Zero to Production-Ready in Minutes (QCon 2017)Netflix: From Zero to Production-Ready in Minutes (QCon 2017)
Netflix: From Zero to Production-Ready in Minutes (QCon 2017)Tim Bozarth
 
Video Conferencing, The Enterprise and You
Video Conferencing, The Enterprise and YouVideo Conferencing, The Enterprise and You
Video Conferencing, The Enterprise and YouVideoguy
 
Production debugging web applications
Production debugging web applicationsProduction debugging web applications
Production debugging web applicationsIdo Flatow
 
From Monoliths to Microservices at Realestate.com.au
From Monoliths to Microservices at Realestate.com.auFrom Monoliths to Microservices at Realestate.com.au
From Monoliths to Microservices at Realestate.com.auevanbottcher
 
Microservices, Microfrontends and Feature Teams
Microservices, Microfrontends and Feature TeamsMicroservices, Microfrontends and Feature Teams
Microservices, Microfrontends and Feature TeamsGiulio Roggero
 
Introducing Zeebe.io at Camunda Meetup Vienna 10/2017
Introducing Zeebe.io at Camunda Meetup Vienna 10/2017Introducing Zeebe.io at Camunda Meetup Vienna 10/2017
Introducing Zeebe.io at Camunda Meetup Vienna 10/2017Daniel Meyer
 
Goto meetup Stockholm - Let your microservices flow
Goto meetup Stockholm - Let your microservices flowGoto meetup Stockholm - Let your microservices flow
Goto meetup Stockholm - Let your microservices flowBernd Ruecker
 
Empower every Azure Function to achieve more!!
Empower every Azure Function to achieve more!!Empower every Azure Function to achieve more!!
Empower every Azure Function to achieve more!!Massimo Bonanni
 
Replay Solutions CFD
Replay Solutions CFDReplay Solutions CFD
Replay Solutions CFDkilroy440
 
The Road To Event-Driven Architecture
The Road To Event-Driven ArchitectureThe Road To Event-Driven Architecture
The Road To Event-Driven ArchitectureSheenBrisals
 
the grinder testing certification
the grinder testing certificationthe grinder testing certification
the grinder testing certificationVskills
 
Js foo - Sept 8 upload
Js foo - Sept 8 uploadJs foo - Sept 8 upload
Js foo - Sept 8 uploadDebnath Sinha
 
Introducing domain driven design - dogfood con 2018
Introducing domain driven design - dogfood con 2018Introducing domain driven design - dogfood con 2018
Introducing domain driven design - dogfood con 2018Steven Smith
 

Similar to Crossroads of Asynchrony and Graceful Degradation (20)

Architecting Microservices in .Net
Architecting Microservices in .NetArchitecting Microservices in .Net
Architecting Microservices in .Net
 
Big datadc skyfall_preso_v2
Big datadc skyfall_preso_v2Big datadc skyfall_preso_v2
Big datadc skyfall_preso_v2
 
Refacoring vs Rewriting WixStores
Refacoring vs Rewriting WixStoresRefacoring vs Rewriting WixStores
Refacoring vs Rewriting WixStores
 
Stress Test as a Culture
Stress Test as a CultureStress Test as a Culture
Stress Test as a Culture
 
Netflix: From Zero to Production-Ready in Minutes (QCon 2017)
Netflix: From Zero to Production-Ready in Minutes (QCon 2017)Netflix: From Zero to Production-Ready in Minutes (QCon 2017)
Netflix: From Zero to Production-Ready in Minutes (QCon 2017)
 
Video Conferencing, The Enterprise and You
Video Conferencing, The Enterprise and YouVideo Conferencing, The Enterprise and You
Video Conferencing, The Enterprise and You
 
Production debugging web applications
Production debugging web applicationsProduction debugging web applications
Production debugging web applications
 
Tec314
Tec314Tec314
Tec314
 
Designing virtual infrastructure
Designing virtual infrastructureDesigning virtual infrastructure
Designing virtual infrastructure
 
From Monoliths to Microservices at Realestate.com.au
From Monoliths to Microservices at Realestate.com.auFrom Monoliths to Microservices at Realestate.com.au
From Monoliths to Microservices at Realestate.com.au
 
Microservices, Microfrontends and Feature Teams
Microservices, Microfrontends and Feature TeamsMicroservices, Microfrontends and Feature Teams
Microservices, Microfrontends and Feature Teams
 
Introducing Zeebe.io at Camunda Meetup Vienna 10/2017
Introducing Zeebe.io at Camunda Meetup Vienna 10/2017Introducing Zeebe.io at Camunda Meetup Vienna 10/2017
Introducing Zeebe.io at Camunda Meetup Vienna 10/2017
 
SOA patterns
SOA patterns SOA patterns
SOA patterns
 
Goto meetup Stockholm - Let your microservices flow
Goto meetup Stockholm - Let your microservices flowGoto meetup Stockholm - Let your microservices flow
Goto meetup Stockholm - Let your microservices flow
 
Empower every Azure Function to achieve more!!
Empower every Azure Function to achieve more!!Empower every Azure Function to achieve more!!
Empower every Azure Function to achieve more!!
 
Replay Solutions CFD
Replay Solutions CFDReplay Solutions CFD
Replay Solutions CFD
 
The Road To Event-Driven Architecture
The Road To Event-Driven ArchitectureThe Road To Event-Driven Architecture
The Road To Event-Driven Architecture
 
the grinder testing certification
the grinder testing certificationthe grinder testing certification
the grinder testing certification
 
Js foo - Sept 8 upload
Js foo - Sept 8 uploadJs foo - Sept 8 upload
Js foo - Sept 8 upload
 
Introducing domain driven design - dogfood con 2018
Introducing domain driven design - dogfood con 2018Introducing domain driven design - dogfood con 2018
Introducing domain driven design - dogfood con 2018
 

More from C4Media

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoC4Media
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileC4Media
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020C4Media
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsC4Media
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No KeeperC4Media
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like OwnersC4Media
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaC4Media
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideC4Media
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDC4Media
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine LearningC4Media
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at SpeedC4Media
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsC4Media
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsC4Media
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerC4Media
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleC4Media
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeC4Media
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereC4Media
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing ForC4Media
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data EngineeringC4Media
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreC4Media
 

More from C4Media (20)

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy Mobile
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java Applications
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No Keeper
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like Owners
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate Guide
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CD
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine Learning
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at Speed
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep Systems
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.js
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly Compiler
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix Scale
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's Edge
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home Everywhere
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing For
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
 

Recently uploaded

From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...Product School
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCzechDreamin
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Product School
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Julian Hyde
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...Product School
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...Sri Ambati
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityScyllaDB
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Product School
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutesconfluent
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backElena Simperl
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka DoktorováCzechDreamin
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxDavid Michel
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Thierry Lestable
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaCzechDreamin
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...CzechDreamin
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
 

Recently uploaded (20)

From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 

Crossroads of Asynchrony and Graceful Degradation

  • 1. Crossroads of asynchrony and graceful degradation Nitesh Kant,! Software Engineer, ! Netflix Edge Engineering.! ! @NiteshKant
  • 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /netflix-asynchronous-apps
  • 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
  • 4.
  • 5.
  • 6.
  • 7. Nitesh Kant Who Am I? ❖ Engineer, Edge Engineering, Netflix.! ❖ Core contributor, RxNetty*! ❖ Contributor, Zuul** * https://github.com/ReactiveX/RxNetty ** https://github.com/Netflix/zuul @NiteshKant
  • 8.
  • 10. A simple example. Showing a movie on Netflix.
  • 14. ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! Disclaimer: This is an example and not an exact representation of the processing
  • 15. Synchronicity ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! Disclaimer: This is an example and not an exact representation of the processing
  • 16. The bigger picture Price of being synchronous? ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! Disclaimer: This is an example and not an exact representation of the processing
  • 17. In a microservices world Edge Service Ratings ServiceVideo Metadata Service Bookmarks Service Disclaimer: This is an example and not an exact representation of the processing
  • 18. In a microservices world Edge Service Server threadpool Thread Thread Thread Thread Thread getMovieMetadata(movieId) Disclaimer: This is an example and not an exact representation of the processing
  • 19. In a microservices world Edge Service Server threadpool Thread Thread Thread Thread Thread getMovieMetadata(movieId) getBookmark(movieId, userId) Disclaimer: This is an example and not an exact representation of the processing
  • 20. In a microservices world Edge Service Server threadpool Thread Thread Thread Thread Thread getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) Disclaimer: This is an example and not an exact representation of the processing
  • 21. Busy thread time = Sum of the time taken to make all 3 service calls
  • 22. How do systems fail? 1. Latency Latency is your worst enemy in a synchronous world.
  • 23. Edge Service Server threadpool Thread Thread Thread Thread Thread getRatings(movieId) Disclaimer: This is an example and not an exact representation of the processing
  • 24. Disclaimer: This is an example and not an exact representation of the processing Ratings Service Edge Service Server threadpool Thread Thread Thread Thread Thread getRatings(movieId)
  • 25. Disclaimer: This is an example and not an exact representation of the processing Edge Service getRatings(movieId) Server threadpool Thread Thread Thread Thread Thread
  • 26. Edge Service Disclaimer: This is an example and not an exact representation of the processing getRatings(movieId) Server threadpool Thread Thread Thread Thread Thread
  • 27. Edge Service Server threadpool Thread Thread Thread Thread Thread Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread getRatings(movieId)
  • 28. Edge Service Server threadpool Thread Thread Thread Thread Thread Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread getRatings(movieId)
  • 29. Edge Service Server threadpool Thread Thread Thread Thread Thread Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread getRatings(movieId)
  • 30. Managing client thread pools Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread
  • 35. Clients have become our babies
  • 36. Clients have become our babies Edge Service Server threadpool Thread Thread Thread Thread Thread getMovieMetadata(movieId) Disclaimer: This is an example and not an exact representation of the processing Client Threadpool Thread Thread Thread Thread Thread Client Threadpool Thread Thread Thread Thread Thread Client Threadpool Thread Thread Thread Thread Thread getBookmark(movieId, userId) getRatings(movieId)
  • 37. Clients have become our babies
  • 39. Have we exchanged a bigger problem with a smaller one?
  • 40. How do systems fail? 2. Overload Abusive clients, recovery spikes, special events ….
  • 41. We did a load test… https://github.com/Netflix-Skunkworks/ WSPerfLab Hello Netflix!
  • 42. Detailed analysis available online: https://github.com/Netflix-Skunkworks/WSPerfLab/blob/master/test-results/RxNetty_vs_Tomcat_April2015.pdf
  • 43.
  • 44. ! Graceful This isn’t graceful degradation!
  • 45. This happens at high CPU usage.
  • 46. This happens at high CPU usage. So, don’t let the system reach that limit…
  • 47. This happens at high CPU usage. So, don’t let the system reach that limit… a.k.a Throttling.
  • 48. Fairness? One abusive request type can penalize other request paths.
  • 49. How do systems fail? 3. Thundering herds The failure after recovery….
  • 50. Retries Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing
  • 51. Retries Edge Service Disclaimer: This is an example and not an exact representation of the processing Video Metadata Service Cluster
  • 52. Retries Edge Service Disclaimer: This is an example and not an exact representation of the processing Video Metadata Service Cluster
  • 53. Retries Edge Service Disclaimer: This is an example and not an exact representation of the processing Video Metadata Service Cluster
  • 54. Retries are useful in steady state…. …but…
  • 55. Retries Edge Service Disclaimer: This is an example and not an exact representation of the processing Video Metadata Service Cluster
  • 56.
  • 57. Our systems are missing empathy.
  • 58. Because they lack knowledge about the peers.
  • 59. Knowledge comes from various signals..
  • 60. Ability to adapt to those signals is important.
  • 61. This can not adapt… ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! Disclaimer: This is an example and not an exact representation of the processing
  • 62. Asynchrony It is the key to success.
  • 63. What should be async?
  • 64. What should be async? Edge Service Video Metadata Service
  • 65. What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) Application logic
  • 66. What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) I/O I/O I/O Application logic I/O
  • 67. What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) I/O I/O I/O Application logic I/O Network protocol
  • 68. Key aspects of being async.
  • 69. Key aspects of being async. 1. Lifecycle control
  • 71. Key aspects of being async. 2. Flow control
  • 73. Key aspects of being async. 3. Function composition
  • 74. Function composition ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }!
  • 75. Function composition Composing the processing ! of a method into a single control point. public Observable<Movie> getMovie(String movieId) {! return Observable.zip(getMovieMetadata(movieId),! getBookmark(movieId, userId),! getRatings(movieId),! (meta,bmark,rating)->new Movie(meta,bmark,rating));! }!
  • 76. Composing the processing ! of a method into a single control point. Flow & Lifecycle Control with
  • 77. What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) I/O I/O I/O Application logic I/O Network protocol
  • 78. I/O Edge Service Server threadpool Thread Thread Thread Thread Thread Client Threadpool Thread Thread Thread Thread Thread getRatings(movieId)
  • 79. I/O Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection Eventloops = f (Number of cores)
  • 80. I/O Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection Connections multiplexed ! on a ! single eventloop.
  • 81. I/O Disclaimer: This is an example and not an exact representation of the processing Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection
  • 82. I/O Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Clients share the eventloops! with! the server.
  • 83. I/O Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection All clients share ! the! same eventloop
  • 84. Composing the processing ! of a service into a single control point. Flow & Lifecycle Control with
  • 85. What should be async? Edge Service Video Metadata Service getMovieMetadata(movieId) getBookmark(movieId, userId) getRatings(movieId) I/O I/O I/O Application logic I/O Network protocol
  • 89. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1
  • 90. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK! ID: 1! …
  • 91. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK! ID: 1! … HTTP/1.1 200 OK! ID: 2! …
  • 92. HTTP/1.1 GET /movie?id=3 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK! ID: 1! … HTTP/1.1 200 OK! ID: 2! … HTTP/1.1 200 OK! ID: 3! …
  • 93. HTTP/1.1 GET /movie?id=3 GET /movie?id=2 GET /movie?id=1 HTTP/1.1 200 ID: 1! … HTTP/1.1 200 ID: 2! … HTTP/1.1 200 ID: 3! … Head Of Line Blocking => Synchronous
  • 95. Network Protocol We need a multiplexed bi-directional protocol
  • 96. Multiplexed GET /movie?id=1 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=3 HTTP/1.1
  • 97. Bi-directional GET /movie?id=1 HTTP/1.1 GET /movie?id=2 HTTP/1.1 GET /movie?id=3 HTTP/1.1 CANCEL
  • 98. Composing the processing ! of the entire application into a single control point. Flow & Lifecycle Control with
  • 99. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing
  • 100. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing Observable<Movie>
  • 101. Observable<Movie> Composing the processing ! of the entire application into a single control point.
  • 103. Latency Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection
  • 104. Latency Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Impact is localized to the connection.
  • 105. Latency Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Impact is localized to the connection. An outstanding request has little cost.
  • 106. An outstanding request has little cost. GET /movie?id=1 HTTP/1.1 HTTP/1.1 200 OK! … } Any stored state between! request - response! is costly.
  • 107. Outstanding requests have low cost so Latency is a lesser evil in asynchronous systems.
  • 108. Overload & Thundering Herds Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Disclaimer: This is an example and not an exact representation of the processing Reduce work done ! when overloaded
  • 109. Overload & Thundering Herds Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Disclaimer: This is an example and not an exact representation of the processing Reduce work done ! when overloaded Stop accepting! new requests.
  • 110. Stop accepting new requests Non-blocking I/O gives better control
  • 111. Stop accepting new requests But … we are still “throttling”
  • 112. Stop accepting new requests Are we being empathetic?
  • 115. Request-leasing Peer 1Peer 2 “Lease” 5 requests for 1 minute. Network connection
  • 116. Request-leasing Peer 1Peer 2 “Lease” 5 requests for 1 minute. GET /movie?id=1 HTTP/1.1 GET /movie?id=2 HTTP/1.1 Network connection
  • 117. Server Client 1 Client 2 Client 8
  • 118. ServerCapacity: 100 RPM Client 1 Client 2 Client 8 “Lease” 10 requests for 1 minute.
  • 119. ServerCapacity: 100 RPM Client 1 Client 2 Client 8 “Lease” 10 requests for 1 minute. “Lease” 10 requests for 1 minute. “Lease” 10 requests for 1 minute.
  • 120. ServerCapacity: 100 RPM Client 1 Client 2 Client 8 “Lease” 10 requests for 1 minute. “Lease” 10 requests for 1 minute. “Lease” 10 requests for 1 minute. Reserve Capacity: ! 20 RPM
  • 121. Time bound lease. “Lease” 10 requests for 1 minute.
  • 122. Time bound lease. No extra work for cancelling leases. “Lease” 10 requests for 1 minute.
  • 123. Time bound lease. No extra work for cancelling leases. Receiver controls the flow of requests “Lease” 10 requests for 1 minute.
  • 124. When things go south
  • 125. ServerCapacity: 20 RPM Client 1 Client 2 Client 8 “Lease” 5 requests for 1 minute. “Lease” 2 requests for 1 minute. X No more “Lease”
  • 126. ServerCapacity: 20 RPM Client 1 Client 2 Client 8 “Lease” 5 requests for 1 minute. “Lease” 2 requests for 1 minute. X No more “Lease” Prioritization
  • 128. Threadpools? Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection I/O! is! non-blocking.
  • 129. Threadpools? Edge Service Disclaimer: This is an example and not an exact representation of the processing Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection Application code ! is! non-blocking.
  • 130. Threadpools? Disclaimer: This is an example and not an exact representation of the processing Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection No blocking/Waiting => Only CPU work
  • 131. Threadpools? Disclaimer: This is an example and not an exact representation of the processing Edge Service Eventloop (Inbound) Connection Connection Connection Connection Connection getMovieMetadata(movieId) Eventloop (Outbound) Connection Connection Connection Connection Connection No blocking/Waiting => Only CPU work So, Eventloops = # of cores
  • 137. Case for timeouts? Read Timeouts Thread Timeouts ✤ Useful in unblocking threads 
 on socket reads. ✤ Business level SLA. ✤ Unblock the calling thread.
  • 138. Case for timeouts? Read Timeouts Thread Timeouts ✤ Useful in unblocking threads 
 on socket reads. ✤ Business level SLA. ✤ Unblock the calling thread. X X As there are no blocking calls.X
  • 139. Case for timeouts? Read Timeouts Thread Timeouts ✤ Useful in unblocking threads 
 on socket reads. ✤ Business level SLA. ✤ Unblock the calling thread.
  • 140. Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing
  • 141. Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store
  • 142. Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store
  • 143. Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store Thread timeouts are pretty invasive at every level
  • 144. Business level SLA Edge Service Video Metadata Service Disclaimer: This is an example and not an exact representation of the processing Rating service C* store C* store Thread timeouts are pretty invasive at every level Do we need them at every step?
  • 145. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing
  • 146. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Business timeouts are ! for ! a client request. Disclaimer: This is an example and not an exact representation of the processing
  • 147. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 Disclaimer: This is an example and not an exact representation of the processing
  • 148. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 X Disclaimer: This is an example and not an exact representation of the processing
  • 149. Edge Service Video Metadata Service Rating service C* store C* store /movie?id=123 X X X X X Disclaimer: This is an example and not an exact representation of the processing
  • 152.
  • 153. Edge Service Video Metadata Service Rating service C* store C* store Disclaimer: This is an example and not an exact representation of the processing Request Leases Cancellations Observable<Movie>
  • 154. Edge Service Video Metadata Service Rating service C* store C* store Disclaimer: This is an example and not an exact representation of the processing Request Leases Cancellations Observable<Movie>
  • 155.
  • 156. ! public Movie getMovie(String movieId) {! Metadata metadata = getMovieMetadata(movieId);! Bookmark bookmark = getBookmark(movieId, userId);! Rating rating = getRatings(movieId);! return new Movie(metadata, bookmark, rating);! }! public Observable<Movie> getMovie(String movieId) {! return Observable.zip(getMovieMetadata(movieId),! getBookmark(movieId, userId),! getRatings(movieId),! (meta,bmark,rating)->new Movie(meta,bmark,rating));! }!
  • 157. Resources Asynchronous Function composition : I/O : Network Protocol : https://github.com/ReactiveX/RxJava https://github.com/ReactiveX/RxNetty http://reactivesocket.io/
  • 158. Nitesh Kant, Engineer, Netflix Edge Gateway @NiteshKant
  • 159. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/netflix- asynchronous-apps