SlideShare a Scribd company logo

Building Fault-Tolerant Systems with Netflix Architectures

•
•
Alex Borysov
Alex Borysov

Slides from ourDevNexus 2020"Break me if you can: a practical guide to building fault-tolerant systems" talk: https://devnexus.com/presentations/4794/

Technology
1 of 181
Break Me If You Can Practical Guide to Building Fault-tolerant Systems DevNexus, Atlanta, GA February 20, 2020 Alex Borysov, Software Engineer @ Netflix Mykyta Protsenko, Software Engineer @ Netflix
Who are we? Alex Borysov Software Engineer @Netflix Mykyta Protsenko Software Engineer @Netflix @aiborisov @mykyta_p @WeAreNetflix
Fault-Tolerance? @aiborisov @mykyta_p
Fault vs Error vs Failure @aiborisov @mykyta_p
@aiborisov @mykyta_p Fault @aiborisov @mykyta_p incorrect internal state Picture by Bob McMillan. Public domain. See slides ##178-181 for details
@aiborisov @mykyta_p Error @aiborisov @mykyta_p visibly incorrect behaviour Picture by David Goehring. CC BY 2.0. See slides ##178-181 for details
@aiborisov @mykyta_p Failure @aiborisov @mykyta_p main functionality is broken Picture by Camerafiend. CC BY-SA 3.0. See slides ##178-181 for details
@aiborisov @mykyta_p RMS Titanic vs Miracle on the Hudson @aiborisov @mykyta_p Willy Stöwer. Public domain. See slides ##178-181 for details By Greg Lam Pak Ng. CC BY 2.0. See slides ##178-181 for details
@aiborisov @mykyta_p RMS Titanic @aiborisov @mykyta_p Fault: Hitting an iceberg Error: Water in the hull Failure: Sinking Willy Stöwer. Public domain. See slides ##178-181 for details
@aiborisov @mykyta_p Miracle on the Hudson @aiborisov @mykyta_p Fault: Hitting geese at 859 m Error: Engines shut down No Failure! By Greg Lam Pak Ng. CC BY 2.0. See slides ##178-181 for details
Fault Error Failure @aiborisov @mykyta_p → →
Fault Error Failure @aiborisov @mykyta_p → →
@aiborisov @mykyta_p Fault Tolerance @aiborisov @mykyta_p Code and Design Patterns Product-Driven Decisions Communication By Greg Lam Pak Ng. CC BY 2.0. See slides ##178-181 for details
Dodging Geese @aiborisov @mykyta_p
@aiborisov @mykyta_p Dodging Geese Architecture TOP-5 Geese Service Clouds Service Leaderboard Service API Gateway @aiborisov @mykyta_p See slides ##178-181 for licensing details
@aiborisov @mykyta_p Dodging Geese Architecture TOP-5 Geese Service Clouds Service Leaderboard Service API Gateway @aiborisov @mykyta_p
@aiborisov @mykyta_p Dodging Geese Architecture TOP-5 Geese Service Leaderboard Service API Gateway @aiborisov @mykyta_p Clouds Service
@aiborisov @mykyta_p Dodging Geese Architecture TOP-5 Leaderboard Service API Gateway @aiborisov @mykyta_p Clouds Service Geese Service
@aiborisov @mykyta_p Dodging Geese Architecture Geese Service Clouds ServiceAPI Gateway @aiborisov @mykyta_p TOP-5 Leaderboard Service
@aiborisov @mykyta_p Dodging Geese Architecture TOP-5 Geese Service Clouds Service Leaderboard Service API Gateway @aiborisov @mykyta_p
@aiborisov @mykyta_p Dodging Geese Architecture TOP-5 Geese Service Clouds Service Leaderboard Service API Gateway @aiborisov @mykyta_p
@aiborisov @mykyta_p Dodging Geese Architecture TOP-5 Geese Service Clouds Service Leaderboard Service API Gateway @aiborisov @mykyta_p
@aiborisov @mykyta_p Leaderboard API (REST) /players/<username>/score {"name": "Jane", "score": 100} /leaderboard/top/<n> [{"name": "Jane", "score": 100}, {"name": "John", "score": 50}, ...] @aiborisov @mykyta_p
@aiborisov @mykyta_p gRPC Service Definitions @aiborisov @mykyta_p service GeeseService { // Return next line of geese. rpc GetGeese (GetGeeseRequest) returns (GeeseResponse); }
@aiborisov @mykyta_p gRPC Service Definitions @aiborisov @mykyta_p service GeeseService { // Return next line of geese. rpc GetGeese (GetGeeseRequest) returns (GeeseResponse); } service CloudsService { // Return next line of clouds. rpc GetClouds (GetCloudsRequest) returns (CloudsResponse); }
@aiborisov @mykyta_p service FixtureService { // Return next line of geese and clouds. rpc GetFixture (GetFixtureRequest) returns (FixtureResponse); } gRPC Gateway Service @aiborisov @mykyta_p
@aiborisov @mykyta_p service FixtureService { // Return next line of geese and clouds. rpc GetFixture (GetFixtureRequest) returns (FixtureResponse); } + = Fixture gRPC Gateway Service @aiborisov @mykyta_p
@aiborisov @mykyta_p public class FixtureService extends FixtureServiceImplBase { Gateway Fixture Service @aiborisov @mykyta_p
@aiborisov @mykyta_p Gateway Fixture Service API Gateway @aiborisov @mykyta_p Geese Service Clouds Service
@aiborisov @mykyta_p @aiborisov @mykyta_p Non-Blocking Calls Don’t block Send requests in parallel Combine results when ready
@aiborisov @mykyta_p public class FixtureService extends FixtureServiceImplBase { Gateway Service Implementation @aiborisov @mykyta_p private final GeeseServiceFutureStub geeseClient = ...; private final CloudsServiceFutureStub cloudsClient = ...;
@aiborisov @mykyta_p public class FixtureService extends FixtureServiceImplBase { Gateway Service Implementation @aiborisov @mykyta_p private final GeeseServiceFutureStub geeseClient = ...; private final CloudsServiceFutureStub cloudsClient = ...; @Override public void getFixture(GetFixtureRequest request, StreamObserver<FixtureResponse> response) { ListenableFuture<GeeseResponse> geese = geeseClient.getGeese(toGeese(request)); ListenableFuture<CloudsResponse> clouds = cloudsClient.getClouds(toClouds(request)); ListenableFuture<List<GeneratedMessageV3>> geeseAndClouds = Futures.allAsList(geese, clouds); ...
@aiborisov @mykyta_p public class FixtureService extends FixtureServiceImplBase { Gateway Service Implementation @aiborisov @mykyta_p private final GeeseServiceFutureStub geeseClient = ...; private final CloudsServiceFutureStub cloudsClient = ...; @Override public void getFixture(GetFixtureRequest request, StreamObserver<FixtureResponse> response) { ListenableFuture<GeeseResponse> geese = geeseClient.getGeese(toGeese(request)); ListenableFuture<CloudsResponse> clouds = cloudsClient.getClouds(toClouds(request)); ListenableFuture<List<GeneratedMessageV3>> geeseAndClouds = Futures.allAsList(geese, clouds); ...
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Slow dependencies Slow upstream services
@aiborisov @mykyta_p @aiborisov @mykyta_p Timeouts Guaranteed latency for integration points
@aiborisov @mykyta_p public class FixtureService extends FixtureServiceImplBase { ... Gateway Service Implementation @aiborisov @mykyta_p @Override public void getFixture(GetFixtureRequest request, StreamObserver<FixtureResponse> response) { ListenableFuture<GeeseResponse> geese = geeseClient.getGeese(toGeese(request)); ListenableFuture<CloudsResponse> clouds = cloudsClient.getClouds(toClouds(request)); ListenableFuture<List<GeneratedMessageV3>> geeseAndClouds = Futures.allAsList(geese, clouds); ...
@aiborisov @mykyta_p public class FixtureService extends FixtureServiceImplBase { ... Gateway Service Implementation @aiborisov @mykyta_p @Override public void getFixture(GetFixtureRequest request, StreamObserver<FixtureResponse> response) { ListenableFuture<GeeseResponse> geese = geeseClient.withDeadlineAfter(500, MILLISECONDS).getGeese(toGeeseRequest(request)); ListenableFuture<CloudsResponse> clouds = cloudsClient.withDeadlineAfter(500, MILLISECONDS).getClouds(toCloudsRequest(request)); ListenableFuture<List<GeneratedMessageV3>> geeseAndClouds = Futures.allAsList(geese, clouds); ...
@aiborisov @mykyta_p @Override public void getFixture(GetFixtureRequest request, StreamObserver<FixtureResponse> response) { ListenableFuture<GeeseResponse> geese = geeseClient.withDeadlineAfter(500, MILLISECONDS).getGeese(toGeeseRequest(request)); ListenableFuture<CloudsResponse> clouds = cloudsClient.withDeadlineAfter(500, MILLISECONDS).getClouds(toCloudsRequest(request)); ListenableFuture<List<GeneratedMessageV3>> geeseAndClouds = Futures.allAsList(geese, clouds); ... public class FixtureService extends FixtureServiceImplBase { ... Gateway Service Implementation @aiborisov @mykyta_p
@aiborisov @mykyta_p REST: Non-Blocking Calls CompletableFuture<List<LeaderboardEntry>> leaderboard = httpClient .get().uri("/top/5") .exchange() .timeout(Duration.ofMillis(500)) .flatMap(cr -> cr.bodyToMono(...)) .toFuture(); @aiborisov @mykyta_p
@aiborisov @mykyta_p REST: Non-Blocking Calls with Timeout CompletableFuture<List<LeaderboardEntry>> leaderboard = httpClient .get().uri("/top/5") .exchange() .timeout(Duration.ofMillis(500)) .flatMap(cr -> cr.bodyToMono(...)) .toFuture(); @aiborisov @mykyta_p
@aiborisov @mykyta_p
Demo @aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p No Geese No Clouds Blinking Leaderboard
@aiborisov @mykyta_p @aiborisov @mykyta_p Observability Monitoring: QPS, latency, errors, ...
@aiborisov @mykyta_p @aiborisov @mykyta_p Observability: gRPC Monitoring: QPS, latency, errors, ... // OpenCensus RpcViews.registerAllViews();
@aiborisov @mykyta_p @aiborisov @mykyta_p Tracing: gRPC GrpcTracing grpcTracing = GrpcTracing.create(...); ManagedChannelBuilder ... .intercept(grpcTracing.newClientInterceptor()) .build() ; ServerBuilder.forPort(8080) ... .intercept(grpcTracing.newServerInterceptor()) .build();
@aiborisov @mykyta_p @aiborisov @mykyta_p Tracing: gRPC GrpcTracing grpcTracing = GrpcTracing.create(...); ManagedChannelBuilder ... .intercept(grpcTracing.newClientInterceptor()) .build(); ServerBuilder.forPort(8080) ... .intercept(grpcTracing.newServerInterceptor()) .build();
@aiborisov @mykyta_p @aiborisov @mykyta_p Tracing: REST build.gradle: dependencies { compile '...:spring-cloud-sleuth-zipkin' compile '...:spring-cloud-starter-sleuth' ... } application.properties: spring.zipkin.baseUrl=http://zipkin:9411/ spring.sleuth.sampler.probability=1.0 spring.sleuth.web.enabled=true
@aiborisov @mykyta_p
Demo @aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Clouds are slow Geese are fast Entire call fails
@aiborisov @mykyta_p ListenableFuture<GeeseResponse> geese = geeseClient..getGeese(toGeese(request)); ListenableFuture<CloudsResponse> clouds = cloudsClient.getClouds(toClouds(request)); ListenableFuture<List<GeneratedMessageV3>> geeseAndClouds = Futures.allAsList(geese, clouds); ... @aiborisov @mykyta_p Partial Degradation
@aiborisov @mykyta_p @aiborisov @mykyta_p Partial Degradation ListenableFuture<GeeseResponse> geese = geeseClient..getGeese(toGeese(request)); ListenableFuture<CloudsResponse> clouds = cloudsClient.getClouds(toClouds(request)); ListenableFuture<List<GeneratedMessageV3>> geeseAndClouds = Futures.successfulAsList(geese, clouds); ...
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Some L-board calls fail L-board latency is low Scores disappear
@aiborisov @mykyta_p CompletableFuture<List<Leaderboard>> request() { return httpClient .get().uri("/top/5").exchange() .timeout(Duration.ofMillis(500)) .flatMap(...).toFuture(); } @aiborisov @mykyta_p Retries: REST
@aiborisov @mykyta_p CompletableFuture<List<Leaderboard>> request() { return httpClient .get().uri("/top/5").exchange() .timeout(Duration.ofMillis(500)) .flatMap(...).toFuture(); } RetryPolicy RETRY_POLICY = new RetryPolicy() .retryOn(IOException.class) .withMaxRetries(MAX_RETRIES); CompletableFuture<List<Leaderboard>> top5 = Failsafe.with(RETRY_POLICY) ... .future(this::httpRequest); @aiborisov @mykyta_p Retries: REST
@aiborisov @mykyta_p
Demo @aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Retry slow calls? Retry failed calls? Retry network faults?
@aiborisov @mykyta_p Retry Storm Clouds ServiceAPI Gateway @aiborisov @mykyta_p
@aiborisov @mykyta_p new RetryPolicy() .withBackoff( MIN_DELAY, MAX_DELAY, TimeUnit.MILLISECONDS, 100.0) ... ... @aiborisov @mykyta_p Exponential Backoffs
@aiborisov @mykyta_p Failsafe .with(RETRY_POLICY) .withFallback( () -> emptyLeaderboard()) ... @aiborisov @mykyta_p Fallbacks
@aiborisov @mykyta_p Failsafe .with(RETRY_POLICY) .withFallback( () -> cachedLeaderboard()) ... @aiborisov @mykyta_p Fallbacks
@aiborisov @mykyta_p Retry Fallback Fail Fast @aiborisov @mykyta_p On Error
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p High 99%ile latency 100 requests Error probability?
@aiborisov @mykyta_p @aiborisov @mykyta_p High 99%ile latency 100 requests Error probability: 1 – 0.99^100 = 63%
@aiborisov @mykyta_p Tail-Tolerance @aiborisov @mykyta_p Request 200 ms deadline
@aiborisov @mykyta_p Tail-Tolerance @aiborisov @mykyta_p Request 200 ms deadline ↓ 100 ms
@aiborisov @mykyta_p Tail-Tolerance @aiborisov @mykyta_p Request 200 ms deadline ↓ 100 ms Request
@aiborisov @mykyta_p Tail-Tolerance @aiborisov @mykyta_p Request 200 ms deadline ↓ 100 ms Request Fastest Response
@aiborisov @mykyta_p High 99%ile latency 100 requests @aiborisov @mykyta_p Request Hedging
@aiborisov @mykyta_p High 99%ile latency 100 requests Error probability: 63% x 0.01 < 1% @aiborisov @mykyta_p Request Hedging
@aiborisov @mykyta_p In gRPC service config "hedgingPolicy": { "maxAttempts": 3, "hedgingDelay": "100ms" } @aiborisov @mykyta_p Hedging in gRPC
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p High mean latency 100 requests Error probability?
@aiborisov @mykyta_p @aiborisov @mykyta_p High mean latency 100 requests Error probability: 1 – 0.50^100 = 99.99...%
@aiborisov @mykyta_p CircuitBreaker CIRCUIT_BREAKER = new CircuitBreaker() .withFailureThreshold(3, 5); CompletableFuture<...> top5 = Failsafe .with(CIRCUIT_BREAKER) .with(RETRY_POLICY) ... .future(this::httpRequest); @aiborisov @mykyta_p Circuit Breaker
@aiborisov @mykyta_p @aiborisov @mykyta_p Error Handling 100% Error Fail Fast Intermittent Slow Hedging Intermittent Fast Retry Fallback✚
@aiborisov @mykyta_p @aiborisov @mykyta_p Error Handling 100% Error Fail Fast Intermittent Slow Hedging Intermittent Fast Retry Fallback✚
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Client-driven deadline Don’t process failed calls
@aiborisov @mykyta_p Deadlines API Gateway @aiborisov @mykyta_p See slides ##178-181 for licensing details
@aiborisov @mykyta_p Deadlines API Gateway @aiborisov @mykyta_p Deadline 500 ms →
@aiborisov @mykyta_p Deadlines API Gateway @aiborisov @mykyta_p Deadline 500 ms → Spent 300 ms →
@aiborisov @mykyta_p Deadlines API Gateway @aiborisov @mykyta_p Spent 300 ms → Spent 250 ms Deadline 500 ms → X
@aiborisov @mykyta_p Deadlines API Gateway @aiborisov @mykyta_p Spent 300 ms → Spent 250 ms Deadline 500 ms → X →
@aiborisov @mykyta_p Deadline Propagation API Gateway @aiborisov @mykyta_p Deadline 500 ms →
@aiborisov @mykyta_p Deadline 200 ms Deadline Propagation API Gateway @aiborisov @mykyta_p Deadline 500 ms → Spent 300 ms →
@aiborisov @mykyta_p Deadline 200 ms Deadline Propagation API Gateway @aiborisov @mykyta_p Spent 300 ms → Spent 250 ms Deadline 500 ms → X
@aiborisov @mykyta_p Deadline 200 ms Deadline Propagation API Gateway @aiborisov @mykyta_p Spent 300 ms → Spent 250 ms Deadline -50 ms Deadline 500 ms → X
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Throughput has limits Exceeding limits?
@aiborisov @mykyta_p new ConcurrencyLimitServletFilter( new ServletLimiterBuilder() .partitionByHeader("GEESE_TYPE", c -> c.assign("premium", 0.9) .assign("free", 0.1)) .limiter(l -> l.limit( newBuilder() .initialLimit(1000)...); @aiborisov @mykyta_p REST
@aiborisov @mykyta_p new ConcurrencyLimitServletFilter( new ServletLimiterBuilder() .partitionByHeader("GEESE_TYPE", c -> c.assign("premium", 0.9) .assign("free", 0.1)) .limiter(l -> l.limit( newBuilder() .initialLimit(1000)...); @aiborisov @mykyta_p REST
@aiborisov @mykyta_p var limiter = new GrpcServerLimiterBuilder() .partitionByHeader(GEESE_TYPE) .partition("premium", 0.9) .partition("free", 0.1) .limiter(l -> l.limit( newBuilder() .initialLimit(1000)...); ConcurrencyLimitServerInterceptor .newBuilder(limiter).build(); @aiborisov @mykyta_p gRPC: Server
@aiborisov @mykyta_p var limiter = new GrpcServerLimiterBuilder() .partitionByHeader(GEESE_TYPE) .partition("premium", 0.9) .partition("free", 0.1) .limiter(l -> l.limit( newBuilder() .initialLimit(1000)...); ConcurrencyLimitServerInterceptor .newBuilder(limiter).build(); @aiborisov @mykyta_p gRPC: Server
@aiborisov @mykyta_p new GrpcClientLimiterBuilder() .limit( newBuilder() .initialLimit(1000).build()) .blockOnLimit(false) // fail-fast .build(); @aiborisov @mykyta_p gRPC: Client
@aiborisov @mykyta_p
Demo @aiborisov @mykyta_p
Demo @aiborisov @mykyta_p
@aiborisov @mykyta_p Monitoring @aiborisov @mykyta_p APM Service metrics Distributed tracing Business metrics Picture by Alex Borysov. CC BY 2.0. See slides ##178-181 for details
@aiborisov @mykyta_p @aiborisov @mykyta_p Code and Design Timeouts / Deadline Propagation Retries / Hedging Proper Fallbacks Concurrency Limits Load Shedding Observability
@aiborisov @mykyta_p @aiborisov @mykyta_p Request for each response Requests don’t change
@aiborisov @mykyta_p Redundant Requests @aiborisov @mykyta_p GeeseRequest GeeseResponse GeeseRequest GeeseResponse GeeseRequest GeeseResponse
@aiborisov @mykyta_p Redundant Requests @aiborisov @mykyta_p GeeseRequest GeeseResponse GeeseRequest GeeseResponse GeeseRequest GeeseResponse
@aiborisov @mykyta_p Streaming @aiborisov @mykyta_p GeeseRequest GeeseResponse GeeseResponse GeeseResponse
@aiborisov @mykyta_p service GeeseService { rpc GetGeese (GetGeeseRequest) returns (GeeseResponse); } service CloudsService { rpc GetClouds (GetCloudsRequest) returns (CloudsResponse); } @aiborisov @mykyta_p gRPC Streaming
@aiborisov @mykyta_p service GeeseService { rpc GetGeese (GetGeeseRequest) returns (stream GeeseResponse); } service CloudsService { rpc GetClouds (GetCloudsRequest) returns (stream CloudsResponse); } @aiborisov @mykyta_p gRPC Streaming
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Server faster than client Client cannot keep up
@aiborisov @mykyta_p Too Many Streaming Responses @aiborisov @mykyta_p GeeseRequest
@aiborisov @mykyta_p Too Many Streaming Responses @aiborisov @mykyta_p GeeseRequest X
@aiborisov @mykyta_p Flow Control @aiborisov @mykyta_p GeeseRequest
@aiborisov @mykyta_p Flow Control @aiborisov @mykyta_p GeeseRequest 5
@aiborisov @mykyta_p Flow Control @aiborisov @mykyta_p GeeseRequest 5
@aiborisov @mykyta_p Flow Control @aiborisov @mykyta_p GeeseRequest 5 3
@aiborisov @mykyta_p Flow Control @aiborisov @mykyta_p GeeseRequest 5 3
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Decople producer and consumer Decople failures
@aiborisov @mykyta_p Message-driven Elastic Responsive Resilient @aiborisov @mykyta_p Reactive Systems
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Per instance limits
@aiborisov @mykyta_p Door Capacity @aiborisov @mykyta_p Why didn’t Rose make room for Jack on the door? Willy Stöwer. Public domain. See slides ##178-181 for details
@aiborisov @mykyta_p Door Capacity @aiborisov @mykyta_p
@aiborisov @mykyta_p Door Capacity @aiborisov @mykyta_p
@aiborisov @mykyta_p Door Capacity @aiborisov @mykyta_p
@aiborisov @mykyta_p Door Capacity @aiborisov @mykyta_p
@aiborisov @mykyta_p Door Capacity @aiborisov @mykyta_p Why didn’t Rose make room for Jack on the door? “ The answer is very simple because it says on page 147 that Jack dies “ James Cameron Willy Stöwer. Public domain. See slides ##178-181 for details
@aiborisov @mykyta_p Capacity @aiborisov @mykyta_p
@aiborisov @mykyta_p Capacity @aiborisov @mykyta_p
@aiborisov @mykyta_p Autoscaling @aiborisov @mykyta_p
@aiborisov @mykyta_p Prescaling @aiborisov @mykyta_p
@aiborisov @mykyta_p Prescaling @aiborisov @mykyta_p See slides ##178-181 for licensing details.
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Services break each other
@aiborisov @mykyta_p $ Free and Premium? Free Premium $
@aiborisov @mykyta_p Free and Premium Outage Free Premium $ $
@aiborisov @mykyta_p $ $ Bulkheads Free Premium $
@aiborisov @mykyta_p Bulkheads Free Premium $ $ $
@aiborisov @mykyta_p @aiborisov @mykyta_p Bulkheads By Request Type By Client Priority By Region By Availability Zone etc
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Deployments can be risky
@aiborisov @mykyta_p Exploding Whale Engineering solution Half a ton of dynamite @aiborisov @mykyta_p Illustration by Greg Williams. CC BY 3.0. See slides ##178-181
@aiborisov @mykyta_p Exploding Whale Engineering solution Half a ton of dynamite Ooops! Non-limited blast radius Learn more at TheExplodingWhale.com @aiborisov @mykyta_p Illustration by Greg Williams. CC BY 3.0. See slides ##178-181
@aiborisov @mykyta_p
Demo @aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Bad user experience Metrics are not enough
@aiborisov @mykyta_p Prober TOP-5 API Gateway @aiborisov @mykyta_p
@aiborisov @mykyta_p Prober TOP-5 API Gateway @aiborisov @mykyta_p See slides ##178-181 for licensing details.
@aiborisov @mykyta_p @aiborisov @mykyta_p Prober Availability Latency SLO Response verification
@aiborisov @mykyta_p @aiborisov @mykyta_p Prober Availability Latency SLO Response verification CloudProber.org
@aiborisov @mykyta_p
@aiborisov @mykyta_p
@aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Technical solutions are not enough
@aiborisov @mykyta_p Communication @aiborisov @mykyta_p
@aiborisov @mykyta_p Communication @aiborisov @mykyta_p
@aiborisov @mykyta_p Communication Channels @aiborisov @mykyta_p GEESE at 270
@aiborisov @mykyta_p Communication Channels @aiborisov @mykyta_p GEESE at 270
@aiborisov @mykyta_p GEESE at 270 Communication Channels @aiborisov @mykyta_p
@aiborisov @mykyta_p GEESE at 270 Communication Channels @aiborisov @mykyta_p
@aiborisov @mykyta_p Postmortems @aiborisov @mykyta_p Blameless Constructive
@aiborisov @mykyta_p Postmortems @aiborisov @mykyta_p Blameless Constructive Social See slides ##178-181 for licensing details
@aiborisov @mykyta_p Postmortems @aiborisov @mykyta_p Timeline Causes Remedies
@aiborisov @mykyta_p @aiborisov @mykyta_p Learn from Failure Blameless postmortems Alert playbooks Incident knowledge base
@aiborisov @mykyta_p
@aiborisov @mykyta_p Libraries and Tools @aiborisov @mykyta_p Demo: github.com/break-me-if-you-can Slides: slides-devnexus.breakit.xyz Failsafe: github.com/jhalterman/failsafe Observability: opencensus.io, opentracing.io Prober: cloudprober.org Concurrency Limits: github.com/Netflix/concurrency-limits
@aiborisov @mykyta_p Demo UI @HalloGene_ Yevgen Golubenko Twitter: @HalloGene_ github.com/HalloGene Picture by Yevgen Golubenko. Also see slides ##178-181 for licensing details
@aiborisov @mykyta_p Books @aiborisov @mykyta_p
@aiborisov @mykyta_p @aiborisov @mykyta_p Fault-Tolerance Code & Design Patterns Product decisions Communication culture
@aiborisov @mykyta_p
@aiborisov @mykyta_p AMA @ the Netflix Booth (#13) @aiborisov @mykyta_p Thursday, Feb 20th 12:50 PM ask Mykyta Protsenko & Alex Borysov anything about fault-tolerance 3:10 PM ask Nadav Cohen anything about developer tooling Friday, Feb 21th 10:00 AM ask Philip Fisher-Ogden anything about Netflix’s streaming architecture 12:50 PM ask Sangeeta Narayanan anything about cloud native services 3:10 PM ask Vinod Viswanathan anything about media processing
@aiborisov @mykyta_p Images and Licensing Images of geese, clouds, pilots, plane, arrows, cup, airport traffic control tower are property of Mykyta Protsenko and Alex Borysov, if not stated otherwise (see below). All Rights Reserved. Other images used: commons.wikimedia.org/wiki/File:FEMA_-_16381_-_Photograph_by_Bob_McMillan_taken_on_09-28-2005_in_Texas.jpg - Picture by Bob McMillan, the US federal government work, public domain www.flickr.com/photos/carbonnyc/3290528875 - Picture by David Goehring. Attribution 2.0 Generic (CC BY 2.0): creativecommons.org/licenses/by/2.0 - changes were made www.flickr.com/photos/carbonnyc/3290528875 - Picture by Camerafiend. Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0): creativecommons.org/licenses/by-sa/3.0/deed.en - no changes were made commons.wikimedia.org/wiki/File:Titanic_sinking,_painting_by_Willy_St%C3%B6wer.jpg - Willy Stöwer. Public domain work of art
@aiborisov @mykyta_p Images and Licensing www.flickr.com/photos/22608787@N00/3200086900 - Picture y Greg Lam Pak Ng. Attribution 2.0 Generic (CC BY 2.0): creativecommons.org/licenses/by/2.0 - no changes were made piq.codeus.net/picture/31994/Blue-Game-Boy-Color - Blue Game Boy Color by kure - Attribution 3.0 Unported (CC BY 3.0): creativecommons.org/licenses/by/3.0 - changes were made piq.codeus.net/picture/191706/The-Sun - The Sun by Vinicius615 - Attribution 3.0 Unported (CC BY 3.0): creativecommons.org/licenses/by/3.0 - changes were made Slide #106: - Picture by Alex Borysov. Attribution 2.0 Generic (CC BY 2.0): creativecommons.org/licenses/by/2.0
@aiborisov @mykyta_p Images and Licensing piq.codeus.net/picture/254492/CVsantahat - Santa hat for CommanderVideo, CVsantahat by anonymous - Attribution 3.0 Unported (CC BY 3.0): creativecommons.org/licenses/by/3.0 - no changes were made piq.codeus.net/picture/423109/UFO - UFO by anonymous - Attribution 3.0 Unported (CC BY 3.0): creativecommons.org/licenses/by/3.0 - no changes were made piq.codeus.net/picture/334023/beer - beer by Investa - Attribution 3.0 Unported (CC BY 3.0): creativecommons.org/licenses/by/3.0 - changes were made
@aiborisov @mykyta_p Images and Licensing piq.codeus.net/picture/444498/Beer-Bottle - Beer Bottle by jacklrj - Attribution 3.0 Unported (CC BY 3.0): creativecommons.org/licenses/by/3.0 - changes were made https://piq.codeus.net/picture/330338/Deal-With-It - Deal With It by Shiro - Attribution 3.0 Unported (CC BY 3.0): creativecommons.org/licenses/by/3.0 - changes were made https://commons.wikimedia.org/wiki/File:Whale_WikiWorld.png - Cartoon illustration has been created by Greg Williams in cooperation with the Wikimedia Foundation - Attribution 3.0 Unported (CC BY 3.0): creativecommons.org/licenses/by/3.0 - changes were made

Recommended

Monitoramento e Gerenciamento de Infraestrutura com Zabbix - Patrícia Ladislau
Monitoramento e Gerenciamento de Infraestrutura com Zabbix - Patrícia LadislauMonitoramento e Gerenciamento de Infraestrutura com Zabbix - Patrícia Ladislau
Monitoramento e Gerenciamento de Infraestrutura com Zabbix - Patrícia LadislauPatricia Ladislau Silva
 
API and Microservices Management
API and Microservices ManagementAPI and Microservices Management
API and Microservices ManagementIBM DataPower Gateway
 
Introduction to WAF and Network Application Security
Introduction to WAF and Network Application SecurityIntroduction to WAF and Network Application Security
Introduction to WAF and Network Application SecurityAlibaba Cloud
 
Cloud-migration-essentials.pdf
Cloud-migration-essentials.pdfCloud-migration-essentials.pdf
Cloud-migration-essentials.pdfALI ANWAR, OCP®
 
Taking control of your queries with GraphQL, Alba Rivas
Taking control of your queries with GraphQL, Alba RivasTaking control of your queries with GraphQL, Alba Rivas
Taking control of your queries with GraphQL, Alba RivasCzechDreamin
 
Solusi UTS Genap Praktikum Sistem Keamanan Jaringan - STMIK Bumigora
Solusi UTS Genap Praktikum Sistem Keamanan Jaringan - STMIK BumigoraSolusi UTS Genap Praktikum Sistem Keamanan Jaringan - STMIK Bumigora
Solusi UTS Genap Praktikum Sistem Keamanan Jaringan - STMIK BumigoraI Putu Hariyadi
 
BGP FlowSpec experience and future developments
BGP FlowSpec experience and future developmentsBGP FlowSpec experience and future developments
BGP FlowSpec experience and future developmentsPavel Odintsov
 
What is APIGEE? What are the benefits of APIGEE?
What is APIGEE? What are the benefits of APIGEE?What is APIGEE? What are the benefits of APIGEE?
What is APIGEE? What are the benefits of APIGEE?IQ Online Training
 

Recommended

Monitoramento e Gerenciamento de Infraestrutura com Zabbix - Patrícia Ladislau
Monitoramento e Gerenciamento de Infraestrutura com Zabbix - Patrícia LadislauMonitoramento e Gerenciamento de Infraestrutura com Zabbix - Patrícia Ladislau
Monitoramento e Gerenciamento de Infraestrutura com Zabbix - Patrícia LadislauPatricia Ladislau Silva
 
API and Microservices Management
API and Microservices ManagementAPI and Microservices Management
API and Microservices ManagementIBM DataPower Gateway
 
Introduction to WAF and Network Application Security
Introduction to WAF and Network Application SecurityIntroduction to WAF and Network Application Security
Introduction to WAF and Network Application SecurityAlibaba Cloud
 
Cloud-migration-essentials.pdf
Cloud-migration-essentials.pdfCloud-migration-essentials.pdf
Cloud-migration-essentials.pdfALI ANWAR, OCP®
 
Taking control of your queries with GraphQL, Alba Rivas
Taking control of your queries with GraphQL, Alba RivasTaking control of your queries with GraphQL, Alba Rivas
Taking control of your queries with GraphQL, Alba RivasCzechDreamin
 
Solusi UTS Genap Praktikum Sistem Keamanan Jaringan - STMIK Bumigora
Solusi UTS Genap Praktikum Sistem Keamanan Jaringan - STMIK BumigoraSolusi UTS Genap Praktikum Sistem Keamanan Jaringan - STMIK Bumigora
Solusi UTS Genap Praktikum Sistem Keamanan Jaringan - STMIK BumigoraI Putu Hariyadi
 
BGP FlowSpec experience and future developments
BGP FlowSpec experience and future developmentsBGP FlowSpec experience and future developments
BGP FlowSpec experience and future developmentsPavel Odintsov
 
What is APIGEE? What are the benefits of APIGEE?
What is APIGEE? What are the benefits of APIGEE?What is APIGEE? What are the benefits of APIGEE?
What is APIGEE? What are the benefits of APIGEE?IQ Online Training
 
How Netflix Is Solving Authorization Across Their Cloud
How Netflix Is Solving Authorization Across Their CloudHow Netflix Is Solving Authorization Across Their Cloud
How Netflix Is Solving Authorization Across Their CloudTorin Sandall
 
Application DoS In Microservice Architectures
Application DoS In Microservice ArchitecturesApplication DoS In Microservice Architectures
Application DoS In Microservice ArchitecturesScott Behrens
 
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...Amazon Web Services
 
Huawei S5700 Basic Configuration Command
Huawei S5700 Basic Configuration CommandHuawei S5700 Basic Configuration Command
Huawei S5700 Basic Configuration CommandHuanetwork
 
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...InfluxData
 
Apache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson LearnedApache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson LearnedGuozhang Wang
 
TechWiseTV Workshop: Cisco TrustSec
TechWiseTV Workshop: Cisco TrustSecTechWiseTV Workshop: Cisco TrustSec
TechWiseTV Workshop: Cisco TrustSecRobb Boyd
 
Netfilter: Making large iptables rulesets scale
Netfilter: Making large iptables rulesets scaleNetfilter: Making large iptables rulesets scale
Netfilter: Making large iptables rulesets scalebrouer
 
Kafka at Peak Performance
Kafka at Peak PerformanceKafka at Peak Performance
Kafka at Peak PerformanceTodd Palino
 
Black and Blue APIs: Attacker's and Defender's View of API Vulnerabilities
Black and Blue APIs: Attacker's and Defender's View of API VulnerabilitiesBlack and Blue APIs: Attacker's and Defender's View of API Vulnerabilities
Black and Blue APIs: Attacker's and Defender's View of API VulnerabilitiesMatt Tesauro
 
Capture the Streams of Database Changes
Capture the Streams of Database ChangesCapture the Streams of Database Changes
Capture the Streams of Database Changesconfluent
 
CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®confluent
 
BGP Advance Technique by Steven & James
BGP Advance Technique by Steven & JamesBGP Advance Technique by Steven & James
BGP Advance Technique by Steven & JamesFebrian ‎
 
F5 DDoS Protection
F5 DDoS ProtectionF5 DDoS Protection
F5 DDoS ProtectionMarketingArrowECS_CZ
 
ISE-CiscoLive.pdf
ISE-CiscoLive.pdfISE-CiscoLive.pdf
ISE-CiscoLive.pdfssuserf4db0a
 
ELK Stack
ELK StackELK Stack
ELK StackPhuc Nguyen
 
Kubernetes networking in AWS
Kubernetes networking in AWSKubernetes networking in AWS
Kubernetes networking in AWSZvika Gazit
 
Apache kafka performance(latency)_benchmark_v0.3
Apache kafka performance(latency)_benchmark_v0.3Apache kafka performance(latency)_benchmark_v0.3
Apache kafka performance(latency)_benchmark_v0.3SANG WON PARK
 
ELK introduction
ELK introductionELK introduction
ELK introductionWaldemar Neto
 
Fortigate Training
Fortigate TrainingFortigate Training
Fortigate TrainingNCS Computech Ltd.
 
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...Alex Borysov
 
Break me if you can: practical guide to building fault-tolerant systems (with...
Break me if you can: practical guide to building fault-tolerant systems (with...Break me if you can: practical guide to building fault-tolerant systems (with...
Break me if you can: practical guide to building fault-tolerant systems (with...Alex Borysov
 

More Related Content

What's hot

How Netflix Is Solving Authorization Across Their Cloud
How Netflix Is Solving Authorization Across Their CloudHow Netflix Is Solving Authorization Across Their Cloud
How Netflix Is Solving Authorization Across Their CloudTorin Sandall
 
Application DoS In Microservice Architectures
Application DoS In Microservice ArchitecturesApplication DoS In Microservice Architectures
Application DoS In Microservice ArchitecturesScott Behrens
 
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...Amazon Web Services
 
Huawei S5700 Basic Configuration Command
Huawei S5700 Basic Configuration CommandHuawei S5700 Basic Configuration Command
Huawei S5700 Basic Configuration CommandHuanetwork
 
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...InfluxData
 
Apache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson LearnedApache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson LearnedGuozhang Wang
 
TechWiseTV Workshop: Cisco TrustSec
TechWiseTV Workshop: Cisco TrustSecTechWiseTV Workshop: Cisco TrustSec
TechWiseTV Workshop: Cisco TrustSecRobb Boyd
 
Netfilter: Making large iptables rulesets scale
Netfilter: Making large iptables rulesets scaleNetfilter: Making large iptables rulesets scale
Netfilter: Making large iptables rulesets scalebrouer
 
Kafka at Peak Performance
Kafka at Peak PerformanceKafka at Peak Performance
Kafka at Peak PerformanceTodd Palino
 
Black and Blue APIs: Attacker's and Defender's View of API Vulnerabilities
Black and Blue APIs: Attacker's and Defender's View of API VulnerabilitiesBlack and Blue APIs: Attacker's and Defender's View of API Vulnerabilities
Black and Blue APIs: Attacker's and Defender's View of API VulnerabilitiesMatt Tesauro
 
Capture the Streams of Database Changes
Capture the Streams of Database ChangesCapture the Streams of Database Changes
Capture the Streams of Database Changesconfluent
 
CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®confluent
 
BGP Advance Technique by Steven & James
BGP Advance Technique by Steven & JamesBGP Advance Technique by Steven & James
BGP Advance Technique by Steven & JamesFebrian ‎
 
Kubernetes networking in AWS
Kubernetes networking in AWSKubernetes networking in AWS
Kubernetes networking in AWSZvika Gazit
 
Apache kafka performance(latency)_benchmark_v0.3
Apache kafka performance(latency)_benchmark_v0.3Apache kafka performance(latency)_benchmark_v0.3
Apache kafka performance(latency)_benchmark_v0.3SANG WON PARK
 

What's hot (20)

How Netflix Is Solving Authorization Across Their Cloud
How Netflix Is Solving Authorization Across Their CloudHow Netflix Is Solving Authorization Across Their Cloud
How Netflix Is Solving Authorization Across Their Cloud
 
Application DoS In Microservice Architectures
Application DoS In Microservice ArchitecturesApplication DoS In Microservice Architectures
Application DoS In Microservice Architectures
 
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...
How Do I Know I Need a Ledger Database? An Intro to Amazon QLDB: re:Invent 20...
 
Huawei S5700 Basic Configuration Command
Huawei S5700 Basic Configuration CommandHuawei S5700 Basic Configuration Command
Huawei S5700 Basic Configuration Command
 
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...
Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...
 
Apache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson LearnedApache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson Learned
 
TechWiseTV Workshop: Cisco TrustSec
TechWiseTV Workshop: Cisco TrustSecTechWiseTV Workshop: Cisco TrustSec
TechWiseTV Workshop: Cisco TrustSec
 
Netfilter: Making large iptables rulesets scale
Netfilter: Making large iptables rulesets scaleNetfilter: Making large iptables rulesets scale
Netfilter: Making large iptables rulesets scale
 
Kafka at Peak Performance
Kafka at Peak PerformanceKafka at Peak Performance
Kafka at Peak Performance
 
Black and Blue APIs: Attacker's and Defender's View of API Vulnerabilities
Black and Blue APIs: Attacker's and Defender's View of API VulnerabilitiesBlack and Blue APIs: Attacker's and Defender's View of API Vulnerabilities
Black and Blue APIs: Attacker's and Defender's View of API Vulnerabilities
 
Capture the Streams of Database Changes
Capture the Streams of Database ChangesCapture the Streams of Database Changes
Capture the Streams of Database Changes
 
CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®
 
BGP Advance Technique by Steven & James
BGP Advance Technique by Steven & JamesBGP Advance Technique by Steven & James
BGP Advance Technique by Steven & James
 
F5 DDoS Protection
F5 DDoS ProtectionF5 DDoS Protection
F5 DDoS Protection
 
ISE-CiscoLive.pdf
ISE-CiscoLive.pdfISE-CiscoLive.pdf
ISE-CiscoLive.pdf
 
ELK Stack
ELK StackELK Stack
ELK Stack
 
Kubernetes networking in AWS
Kubernetes networking in AWSKubernetes networking in AWS
Kubernetes networking in AWS
 
Apache kafka performance(latency)_benchmark_v0.3
Apache kafka performance(latency)_benchmark_v0.3Apache kafka performance(latency)_benchmark_v0.3
Apache kafka performance(latency)_benchmark_v0.3
 
ELK introduction
ELK introductionELK introduction
ELK introduction
 
Fortigate Training
Fortigate TrainingFortigate Training
Fortigate Training
 

Similar to Building Fault-Tolerant Systems with Netflix Architectures

OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...Alex Borysov
 
Break me if you can: practical guide to building fault-tolerant systems (with...
Break me if you can: practical guide to building fault-tolerant systems (with...Break me if you can: practical guide to building fault-tolerant systems (with...
Break me if you can: practical guide to building fault-tolerant systems (with...Alex Borysov
 
Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...
Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...
Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...Alex Borysov
 
CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...
CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...
CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...Alex Borysov
 
Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...
Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...
Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...Alex Borysov
 
"gRPC vs REST: let the battle begin!" OSCON 2018 edition
"gRPC vs REST: let the battle begin!" OSCON 2018 edition"gRPC vs REST: let the battle begin!" OSCON 2018 edition
"gRPC vs REST: let the battle begin!" OSCON 2018 editionAlex Borysov
 
"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition
"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition
"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 editionAlex Borysov
 
"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition
"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition
"gRPC vs REST: let the battle begin!" DevoxxUK 2018 editionAlex Borysov
 
gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!Alex Borysov
 
gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!Alex Borysov
 
3h à l'assaut d'une application réactive - Devoxx 2015
3h à l'assaut d'une application réactive - Devoxx 20153h à l'assaut d'une application réactive - Devoxx 2015
3h à l'assaut d'une application réactive - Devoxx 2015Publicis Sapient Engineering
 
Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?
Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?
Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?Alex Borysov
 
Thinkful DC - Intro to JavaScript
Thinkful DC - Intro to JavaScriptThinkful DC - Intro to JavaScript
Thinkful DC - Intro to JavaScriptTJ Stalcup
 
Managing microservices with istio on OpenShift - Meetup
Managing microservices with istio on OpenShift - MeetupManaging microservices with istio on OpenShift - Meetup
Managing microservices with istio on OpenShift - MeetupJosé Román Martín Gil
 
Baremetal deployment scale
Baremetal deployment scaleBaremetal deployment scale
Baremetal deployment scalebaremetal
 
JavaFest. Cedrick Lunven. Build APIS with SpringBoot - REST, GRPC, GRAPHQL wh...
JavaFest. Cedrick Lunven. Build APIS with SpringBoot - REST, GRPC, GRAPHQL wh...JavaFest. Cedrick Lunven. Build APIS with SpringBoot - REST, GRPC, GRAPHQL wh...
JavaFest. Cedrick Lunven. Build APIS with SpringBoot - REST, GRPC, GRAPHQL wh...FestGroup
 
Baremetal deployment
Baremetal deploymentBaremetal deployment
Baremetal deploymentbaremetal
 
apidays LIVE Australia - From micro to macro-coordination through domain-cent...
apidays LIVE Australia - From micro to macro-coordination through domain-cent...apidays LIVE Australia - From micro to macro-coordination through domain-cent...
apidays LIVE Australia - From micro to macro-coordination through domain-cent...apidays
 
Cleaning your architecture with android architecture components
Cleaning your architecture with android architecture componentsCleaning your architecture with android architecture components
Cleaning your architecture with android architecture componentsDebora Gomez Bertoli
 
Intro to Javascript
Intro to JavascriptIntro to Javascript
Intro to JavascriptTJ Stalcup
 

Similar to Building Fault-Tolerant Systems with Netflix Architectures (20)

OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
 
Break me if you can: practical guide to building fault-tolerant systems (with...
Break me if you can: practical guide to building fault-tolerant systems (with...Break me if you can: practical guide to building fault-tolerant systems (with...
Break me if you can: practical guide to building fault-tolerant systems (with...
 
Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...
Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...
Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...
 
CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...
CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...
CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...
 
Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...
Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...
Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...
 
"gRPC vs REST: let the battle begin!" OSCON 2018 edition
"gRPC vs REST: let the battle begin!" OSCON 2018 edition"gRPC vs REST: let the battle begin!" OSCON 2018 edition
"gRPC vs REST: let the battle begin!" OSCON 2018 edition
 
"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition
"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition
"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition
 
"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition
"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition
"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition
 
gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!
 
gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!
 
3h à l'assaut d'une application réactive - Devoxx 2015
3h à l'assaut d'une application réactive - Devoxx 20153h à l'assaut d'une application réactive - Devoxx 2015
3h à l'assaut d'une application réactive - Devoxx 2015
 
Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?
Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?
Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?
 
Thinkful DC - Intro to JavaScript
Thinkful DC - Intro to JavaScriptThinkful DC - Intro to JavaScript
Thinkful DC - Intro to JavaScript
 
Managing microservices with istio on OpenShift - Meetup
Managing microservices with istio on OpenShift - MeetupManaging microservices with istio on OpenShift - Meetup
Managing microservices with istio on OpenShift - Meetup
 
Baremetal deployment scale
Baremetal deployment scaleBaremetal deployment scale
Baremetal deployment scale
 
JavaFest. Cedrick Lunven. Build APIS with SpringBoot - REST, GRPC, GRAPHQL wh...
JavaFest. Cedrick Lunven. Build APIS with SpringBoot - REST, GRPC, GRAPHQL wh...JavaFest. Cedrick Lunven. Build APIS with SpringBoot - REST, GRPC, GRAPHQL wh...
JavaFest. Cedrick Lunven. Build APIS with SpringBoot - REST, GRPC, GRAPHQL wh...
 
Baremetal deployment
Baremetal deploymentBaremetal deployment
Baremetal deployment
 
apidays LIVE Australia - From micro to macro-coordination through domain-cent...
apidays LIVE Australia - From micro to macro-coordination through domain-cent...apidays LIVE Australia - From micro to macro-coordination through domain-cent...
apidays LIVE Australia - From micro to macro-coordination through domain-cent...
 
Cleaning your architecture with android architecture components
Cleaning your architecture with android architecture componentsCleaning your architecture with android architecture components
Cleaning your architecture with android architecture components
 
Intro to Javascript
Intro to JavascriptIntro to Javascript
Intro to Javascript
 

More from Alex Borysov

"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019
"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019
"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019Alex Borysov
 
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019Alex Borysov
 
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019Alex Borysov
 
"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition
"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition
"Enabling Googley microservices with gRPC" Riga DevDays 2018 editionAlex Borysov
 
"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition
"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition
"Enabling Googley microservices with gRPC" VoxxedDays Minsk editionAlex Borysov
 
"Enabling Googley microservices with gRPC" at JDK.IO 2017
"Enabling Googley microservices with gRPC" at JDK.IO 2017"Enabling Googley microservices with gRPC" at JDK.IO 2017
"Enabling Googley microservices with gRPC" at JDK.IO 2017Alex Borysov
 
"Enabling Googley microservices with gRPC" at JEEConf 2017
"Enabling Googley microservices with gRPC" at JEEConf 2017"Enabling Googley microservices with gRPC" at JEEConf 2017
"Enabling Googley microservices with gRPC" at JEEConf 2017Alex Borysov
 
"Enabling Googley microservices with gRPC." at Devoxx France 2017
"Enabling Googley microservices with gRPC." at Devoxx France 2017"Enabling Googley microservices with gRPC." at Devoxx France 2017
"Enabling Googley microservices with gRPC." at Devoxx France 2017Alex Borysov
 
Enabling Googley microservices with HTTP/2 and gRPC.
Enabling Googley microservices with HTTP/2 and gRPC.Enabling Googley microservices with HTTP/2 and gRPC.
Enabling Googley microservices with HTTP/2 and gRPC.Alex Borysov
 

More from Alex Borysov (9)

"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019
"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019
"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019
 
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
 
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
 
"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition
"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition
"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition
 
"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition
"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition
"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition
 
"Enabling Googley microservices with gRPC" at JDK.IO 2017
"Enabling Googley microservices with gRPC" at JDK.IO 2017"Enabling Googley microservices with gRPC" at JDK.IO 2017
"Enabling Googley microservices with gRPC" at JDK.IO 2017
 
"Enabling Googley microservices with gRPC" at JEEConf 2017
"Enabling Googley microservices with gRPC" at JEEConf 2017"Enabling Googley microservices with gRPC" at JEEConf 2017
"Enabling Googley microservices with gRPC" at JEEConf 2017
 
"Enabling Googley microservices with gRPC." at Devoxx France 2017
"Enabling Googley microservices with gRPC." at Devoxx France 2017"Enabling Googley microservices with gRPC." at Devoxx France 2017
"Enabling Googley microservices with gRPC." at Devoxx France 2017
 
Enabling Googley microservices with HTTP/2 and gRPC.
Enabling Googley microservices with HTTP/2 and gRPC.Enabling Googley microservices with HTTP/2 and gRPC.
Enabling Googley microservices with HTTP/2 and gRPC.
 

Recently uploaded

Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?XfilesPro
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 

Recently uploaded (20)

Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 

Building Fault-Tolerant Systems with Netflix Architectures