Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Enabling Googley microservices with HTTP/2 and gRPC.

13,730 views

Published on

Slides from my #JavaDay2016 talk "Enabling Googley microservices with HTTP/2 and gRPC.
A high performance, open source, HTTP/2-based RPC framework."

Published in: Software
  • how can i download this slide ?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Enabling Googley microservices with HTTP/2 and gRPC.

  1. 1. Alex Borysov Software Engineer Enabling Googley microservices with HTTP/2 and gRPC. A high performance, open source, HTTP/2-based RPC framework.
  2. 2. Google Cloud Platform Alex Borysov • Software engineer / technical leader experienced in large scale software development. • 10+ years of software engineering experience. • Active gRPC user. Software Engineer @ Google @aiborisov
  3. 3. Google Cloud Platform Tens of billions requests per second. Google Scale
  4. 4. Google Cloud Platform Why gRPC? Photo taken by Andrey Borisenko.
  5. 5. Google Cloud Platform gRPC vs JSON/HTTP for Google Cloud Pub/Sub Publishing 50KB messages at maximum throughput from a single n1-highcpu-16 GPE VM instance, using 9 gRPC channels. More impressive than the almost 3x increase in throughput, is that it took only 1/4 of the CPU resources. 11x difference per CPU3x increase in throughput https://cloud.google.com/blog/big-data/2016/03/announcing-grpc-alpha-for-google-cloud-pubsub
  6. 6. Google Cloud Platform What is gRPC? gRPC stands for gRPC Remote Procedure Calls. A high performance, open source, general purpose standards-based, feature-rich RPC framework. Open sourced version of Stubby RPC used in Google. Actively developed and production-ready, current version is 1.0.1.
  7. 7. Google Cloud Platform 1990s vs 2016 1997 2016 Home Internet Dial-up connection 22,8 Kbps, 33.6Kbps, 56Kbps Fiber optic 100 - 1,000Mbps # of active web sites ~ 1.1 million ~ 1 billion Internet access Wired PCs PCs, laptops, mobiles, IoT, etc. Internet sites Text and image pages Multimedia applications
  8. 8. Google Cloud Platform 1997 vs 2016 - World Wide Web Consortium
  9. 9. Google Cloud Platform History of HTTP 1991 1993 1995 1997 1999 HTTP/0.9 2001 2003 2005 2007 2009 2011 2013 HTTP/1.0 HTTP/1.1 2015 2017 ?
  10. 10. Google Cloud Platform HTTP/1.1 Initially released in 1997 (RFC 2068) Updated in June 1999 (RFC 2616) More updates in 2014 (RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235)
  11. 11. 11 New TCP connection per HTTP connection. # of parallel HTTP requests = # of TCP connections. SSL / TLS overhead. H1 Limited Parallelism
  12. 12. 12 H1 Head-of-line Blocking
  13. 13. 13 H1 Head-of-line Blocking HTTP/1.1 HOL blocking
  14. 14. 14 HTTP metadata (headers). Not compressed plain text headers for each and every HTTP request. H1 Protocol Overhead
  15. 15. 15 HTTP metadata (headers). Not compressed plain text headers for each and every HTTP request. Workarounds: cookies, sessions, concatenation, sprinting, etc. H1 Protocol Overhead
  16. 16. 16 Limited parallelism. Head-of-line blocking. High protocol overhead. HTTP/1.x Photo taken by Andrey Borisenko.
  17. 17. Google Cloud Platform HTTP/2 Released 2015. First new version of HTTP since HTTP/1.1, published in RFC 2068 in 1997.
  18. 18. Google Cloud Platform HTTP/2 Released 2015. First new version of HTTP since HTTP/1.1, published in RFC 2068 in 1997. Design goals: • Reduce latency. • Minimize protocol overhead. • Add support for request prioritization • Add support for server push.
  19. 19. Google Cloud Platform HTTP/2 HTTP/2 is extending, not replacing, the previous HTTP standards. The application semantics of HTTP are the same: • HTTP header fields • HTTP Methods • Request-response • Status codes • URIs
  20. 20. Google Cloud Platform HTTP/2 HTTP/2 is extending, not replacing, the previous HTTP standards. The application semantics of HTTP are the same: • HTTP header fields • HTTP Methods • Request-response • Status codes • URIs HTTP/2 modifies how the data is formatted (framed) and transported between the client and server.
  21. 21. Google Cloud Platform HTTP/2 HTTP/2 is extending, not replacing, the previous HTTP standards. The application semantics of HTTP are the same:: • HTTP header fields • HTTP Methods • Request-response • Status codes • URIs HTTP/2 modifies how the data is formatted (framed) and transported between the client and server.
  22. 22. Google Cloud Platform • Single TCP connection. • No Head-of-line blocking. • Binary framing layer. • Request –> Stream. • Header Compression. HTTP/2 in One Slide Transport(TCP) Application (HTTP/2) Network (IP) Session (TLS) [optional] Binary Framing HEADERS Frame DATA Frame HTTP/2 POST: /upload HTTP/1.1 Host: www.javaday.org.ua Content-Type: application/json Content-Length: 27 HTTP/1.x {“msg”: “Welcome to 2016!”}
  23. 23. Google Cloud Platform HTTP/2 Binary Framing • Stream is a bidirectional flow of bytes within an established connection, which may carry one or more messages. • Message is a complete sequence of frames that map to a logical request or response message. • Frame is the smallest unit of communication in HTTP/2, each containing a frame header, which at a minimum identifies the stream to which the frame belongs: HEADERS for metadata, DATA for payload, RST_STREAM SETTINGS, PUSH_PROMISE, PING, GOAWAY, WINDOW_UPDATE, etc.
  24. 24. Google Cloud Platform HTTP/2 breaks down the HTTP protocol communication into an exchange of binary-encoded frames, which are then mapped to messages that belong to a stream, and all of which are multiplexed within a single TCP connection. Binary Framing Stream 1 HEADERS Stream 2 :method: GET :path: /kyiv :version: HTTP/2 :scheme: https HEADERS :status: 200 :version: HTTP/2 :server: nginx/1.10.1 ... DATA <payload> Stream N Request Response TCP
  25. 25. Google Cloud Platform HTTP/2 Request and Response Multiplexing Interleave multiple requests and responses in parallel without blocking on any one. Use a single TCP connection to deliver multiple requests and responses in parallel. Enable flow-control, server push, etc. Stream 1 HEADERS Stream 2 DATA Stream 3 HEADERS Stream 3 DATA Stream 1 DATA Stream Y HEADERS Stream X DATA Requests Responses HTTP/2 connection Client Server
  26. 26. 26 Client and server maintain and update an indexed list of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value Request #1
  27. 27. 27 Client and server maintain and update an indexed list of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value Request #1 :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value HEADERS Frame
  28. 28. 28 Client and server maintain and update an indexed list of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value :method GET :scheme HTTPS :host myhost.com :path /resource custom_header some_value Request #1 Request #2 :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value HEADERS Frame
  29. 29. 29 Client and server maintain and update an indexed list of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value :method GET :scheme HTTPS :host myhost.com :path /resource custom_header some_value Request #1 Request #2 :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value HEADERS Frame
  30. 30. 30 Client and server maintain and update an indexed list of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value :method GET :scheme HTTPS :host myhost.com :path /resource custom_header some_value Request #1 Request #2 :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value HEADERS Frame :path /resource + indexes for already seen values HEADERS Frame
  31. 31. 31 Static table with 61 common values. • single byte 0x82 means add “:method” == “GET” Dynamic table is initially empty, updated based on sent values. Values are encoded with a static Huffman code. HPACK
  32. 32. Google Cloud Platform HTTP/1.x vs HTTP/2 http://http2.golang.org/gophertiles http://www.http2demo.io/
  33. 33. Google Cloud Platform HTTP/1.x vs HTTP/2
  34. 34. Google Cloud Platform HTTP/1.x vs HTTP/2 6 connections 1 connection
  35. 35. Google Cloud Platform HTTP/2 in the Wild • Apache Tomcat 8.5+ • Apache HTTP Server 2.4.17+ (C) • NGINX (C) • Jetty • Netty • Undertow • Vert.x • OkHttp (Android) • and more • curl 7.43.0+ • Wireshark • jmeter • HTTP/2 Test • h2i • h2load • and more • Chrome • Firefox • Safari* • Opera • Edge • IE 11* • Android Browser • Chrome for Android • iOS Safari *only HTTP/2 over TLS (https) Browsers • JEP 110: HTTP/2 Client. • JEP 244: TLS ALPN Extension. JavaImplementations Tools https://github.com/http2/http2-spec/wiki/Implementations https://github.com/http2/http2-spec/wiki/Tools http://caniuse.com/#feat=http2
  36. 36. Google Cloud Platform History of HTTP 1991 1993 1995 1997 1999 HTTP/0.9 2001 2003 2005 2007 2009 2011 2013 HTTP/1.0 HTTP/1.1 2015 2017
  37. 37. Google Cloud Platform History of HTTP 1991 1993 1995 1997 1999 HTTP/0.9 2001 2003 2005 2007 2009 2011 2013 HTTP/1.0 HTTP/1.1 SPDY HTTP/2.0 2015 2017
  38. 38. Google Cloud Platform gRPC and HTTP/2 Google SPDY –> HTTP/2. Google Stubby –> gRPC.
  39. 39. Google Cloud Platform gRPC and HTTP/2 Google SPDY –> HTTP/2. Google Stubby –> gRPC. gRPC is an HTTP/2-based RPC framework. Because performance matters. Cloud Pub/Sub: 11x throughput improvement [per CPU].
  40. 40. Google Cloud Platform gRPC and HTTP/2 Google SPDY –> HTTP/2. Google Stubby –> gRPC. gRPC is an HTTP/2-based RPC framework. Because performance matters. Cloud Pub/Sub: 11x throughput improvement [per CPU].
  41. 41. Google Cloud Platform How?
  42. 42. Google Cloud Platform Service Definition
  43. 43. Google Cloud Platform Service Definition (weather.proto)
  44. 44. Google Cloud Platform Service Definition (weather.proto) syntax = "proto3"; service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); }
  45. 45. Google Cloud Platform Service Definition syntax = "proto3"; option java_multiple_files = true; option java_package = "ua.org.javaday.grpcexample"; package grpcexample; service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); }
  46. 46. Google Cloud Platform Service Definition - Request Message message WeatherRequest { Coordinates coordinates = 1; message Coordinates { fixed64 latitude = 1; fixed64 longitude = 2; } }
  47. 47. Google Cloud Platform Service Definition - Response Message message WeatherResponse { Temperature temperature = 1; float humidity = 2; } message Temperature { float degrees = 1; Units units = 2; enum Units { FAHRENHEIT = 0; CELSIUS = 1; KELVIN = 2; } }
  48. 48. Google Cloud Platform Service Definition (weather.proto) syntax = "proto3"; service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); } message WeatherRequest { Coordinates coordinates = 1; message Coordinates { fixed64 latitude = 1; fixed64 longitude = 2; } } message WeatherResponse { Temperature temperature = 1; float humidity = 2; } message Temperature { float degrees = 1; Units units = 2; enum Units { FAHRENHEIT = 0; CELSIUS = 1; KELVIN = 2; } }
  49. 49. Google Cloud Platform Generated Classes $ ls -1 build/generated/source/proto/main/java/ua/org/javaday/grpcexample/ Temperature.java TemperatureOrBuilder.java WeatherProto.java WeatherRequest.java WeatherRequestOrBuilder.java WeatherResponse.java WeatherResponseOrBuilder.java
  50. 50. Google Cloud Platform Generated Classes $ ls -1 build/generated/source/proto/main/java/ua/org/javaday/grpcexample/ Temperature.java TemperatureOrBuilder.java WeatherProto.java WeatherRequest.java WeatherRequestOrBuilder.java WeatherResponse.java WeatherResponseOrBuilder.java $ ls -1 build/generated/source/proto/main/grpc/ua/org/javaday/grpcexample/ WeatherGrpc.java
  51. 51. Google Cloud Platform First Law of Distributed Object Design: don't distribute your objects. Martin Fowler http://martinfowler.com/articles/distributed-objects-microservices.html “ ”
  52. 52. Google Cloud Platform The Fallacies of Distributed Computing The network is reliable Latency is zero Bandwidth is infinite The network is secure https://blogs.oracle.com/jag/resource/Fallacies.html Topology doesn't change There is one administrator Transport cost is zero The network is homogeneous
  53. 53. Google Cloud Platform Services not Objects, Messages not References. Promote the microservices design philosophy of coarse-grained message exchange between systems while avoiding the pitfalls of distributed objects and the fallacies of ignoring the network. gRPC Principles & Requirements http://www.grpc.io/blog/principles
  54. 54. Google Cloud Platform Implement gRPC Server public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { } }
  55. 55. Google Cloud Platform Implement gRPC Server public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { } } Abstract class!
  56. 56. Google Cloud Platform Implement gRPC Server public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { WeatherResponse response = WeatherResponse.newBuilder() .setTemperature(Temperature.newBuilder().setDegrees(70).setUnits(FAHRENHEIT)) .setHumidity(.65f).build(); responseObserver.onNext(response); responseObserver.onCompleted(); } }
  57. 57. Google Cloud Platform Start gRPC Server void start() throws IOException { Server grpcServer = NettyServerBuilder.forPort(8090) .addService(new WeatherService()).build() .start(); Runtime.getRuntime().addShutdownHook(new Thread(grpcServer::shutdown)); grpcServer.awaitTerminaton(); }
  58. 58. Google Cloud Platform gRPC Clients ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build(); WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel);
  59. 59. Google Cloud Platform gRPC Clients ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build(); WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel); WeatherGrpc.WeatherBlockingStub blockingClient = WeatherGrpc.newBlockingStub(grpcChannel);
  60. 60. Google Cloud Platform gRPC Clients ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build(); WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel); WeatherGrpc.WeatherBlockingStub blockingClient = WeatherGrpc.newBlockingStub(grpcChannel); WeatherGrpc.WeatherFutureStub futureClient = WeatherGrpc.newFutureStub(grpcChannel);
  61. 61. Google Cloud Platform gRPC Async Client WeatherRequest request = WeatherRequest.newBuilder() .setCoordinates(Coordinates.newBuilder().setLatitude(420000000) .setLongitude(-720000000)).build(); client.getCurrent(request, new StreamObserver<WeatherResponse>() { @Override public void onNext(WeatherResponse response) { logger.info("Current weather for {}: {}", request, response); } @Override public void onError(Throwable t) { logger.info("Cannot get weather for {}", request); } @Override public void onCompleted() { logger.info("Stream completed."); } });
  62. 62. Google Cloud Platform gRPC Sync Client WeatherRequest request = WeatherRequest.newBuilder() .setCoordinates(Coordinates.newBuilder().setLatitude(420000000) .setLongitude(-720000000)).build(); WeatherResponse response = blockingClient.getCurrent(request); logger.info("Current weather for {}: {}", request, response);
  63. 63. Google Cloud Platform gRPC ListenableFuture Async Client WeatherRequest request = WeatherRequest.newBuilder() .setCoordinates(Coordinates.newBuilder().setLatitude(420000000) .setLongitude(-720000000)).build(); ListenableFuture<WeatherResponse> future = futureClient.getCurrent(request);
  64. 64. Google Cloud Platform gRPC ListenableFuture Async Client ListenableFuture<WeatherResponse> future = futureClient.getCurrent(request); Futures.addCallback(future, new FutureCallback<WeatherResponse>() { @Override public void onSuccess(@Nullable WeatherResponse result) { logger.info("Current weather for {}: {}", request, response); } @Override public void onFailure(Throwable t) { logger.info("Cannot get current weather for {}", request); } });
  65. 65. Google Cloud Platform gRPC ListenableFuture Async Client WeatherRequest request = WeatherRequest.newBuilder() .setCoordinates(Coordinates.newBuilder().setLatitude(420000000) .setLongitude(-720000000)).build(); Futures.transformAsync(futureClient.getCurrent(request), response -> { logger.info("Current weather for {}: {}", request, response); return ...; });
  66. 66. Google Cloud Platform Service Definition - Response Message message WeatherResponse { Temperature temperature = 1; float humidity = 2; }
  67. 67. Google Cloud Platform Adding New Field message WeatherResponse { Temperature temperature = 1; float humidity = 2; Wind wind = 3; } message Wind { Speed speed = 1; float direction = 2; } message Speed { float value = 1; Units units = 2; enum Units { MPH = 0; MPS = 1; KNOTS = 2; KMH = 3; } }
  68. 68. Google Cloud Platform Adding New [Micro]Services Weather Weather Temp Wind Humidity
  69. 69. Google Cloud Platform Service Definitions service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); } service TemperatureService { rpc GetCurrent(Coordinates) returns (Temperature); } service HumidityService { rpc GetCurrent(Coordinates) returns (HumidityResponse); } service WindService { rpc GetCurrent(Coordinates) returns (Wind); }
  70. 70. Google Cloud Platform Blocking Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { private final TemperatureServiceBlockingStub temperatureService; private final HumidityServiceBlockingStub humidityService; private final WindServiceBlockingStub windService; public WeatherService(TemperatureServiceBlockingStub temperatureService, HumidityServiceBlockingStub humidityService, WindServiceBlockingStub windService) { this.temperatureService = temperatureService; this.humidityService = humidityService; this.windService = windService; } ...
  71. 71. Google Cloud Platform Blocking Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Temperature temperature = temperatureService.getCurrent(request.getCoordinates()); HumidityResponse humidity = humidityService.getCurrent(request.getCoordinates()); Wind wind = windService.getCurrent(request.getCoordinates()); WeatherResponse response = WeatherResponse.newBuilder().setTemperature(temperature) .setHumidity(humidity.getValue()).setWind(wind).build(); responseObserver.onNext(response); responseObserver.onCompleted(); } }
  72. 72. 72 Blocking Stubs Temp Humidity Wind
  73. 73. Google Cloud Platform Async Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { private final TemperatureServiceFutureStub tempService; private final HumidityServiceFutureStub humidityService; private final WindServiceFutureStub windService; public WeatherService(TemperatureServiceFutureStub tempService, HumidityServiceFutureStub humidityService, WindServiceFutureStub windService) { this.tempService = tempService; this.humidityService = humidityService; this.windService = windService; } ...
  74. 74. Google Cloud Platform Async Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Coordinates coordinates = request.getCoordinates(); Builder responseBuilder = WeatherResponse.newBuilder(); ListenableFuture<List<Builder>> responsesFuture = Futures.allAsList( Futures.transform(tempService.getCurrent(coordinates), responseBuilder::setTemperature), Futures.transform(humidityService.getCurrent(coordinates), (HumidityResponse humidity) -> responseBuilder.setHumidity(humidity.getValue())), Futures.transform(windService.getCurrent(coordinates), responseBuilder::setWind) ); ...
  75. 75. Google Cloud Platform Async Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ... Futures.addCallback(responsesFuture, new FutureCallback<List<Builder>>() { @Override public void onSuccess(@Nullable List<Builder> result) { responseObserver.onNext(responseBuilder.build()); responseObserver.onCompleted(); } @Override public void onFailure(Throwable t) { responseObserver.onError(t);} }); } }
  76. 76. Google Cloud Platform Async Stub Dependencies ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build(); WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel); WeatherGrpc.WeatherFutureStub futureClient = WeatherGrpc.newFutureStub(grpcChannel);
  77. 77. Google Cloud Platform Netty Transport ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build(); WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel); WeatherGrpc.WeatherFutureStub futureClient = WeatherGrpc.newFutureStub(grpcChannel); Server grpcServer = NettyServerBuilder.forPort(8090) .addService(new WeatherService()).build().start();
  78. 78. Google Cloud Platform Netty: Asynchronous and Non-Blocking IO Netty-based transport: • Multiplexes conn-s on the event loops: EpollEventLoopGroup, NioEventLoopGroup. • Decouples I/O waiting from threads. • gRPC uses Netty event loops for both client and server transports. Request Event Loop Worker thread Worker thread Worker thread Worker thread Worker thread Worker thread Worker thread Event Loop Event Loop Event Loop Callback Request Callback N e t w o r k N e t w o r k
  79. 79. Google Cloud Platform Async Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ... Futures.addCallback(responsesFuture, new FutureCallback<List<Builder>>() { @Override public void onSuccess(@Nullable List<Builder> result) { responseObserver.onNext(responseBuilder.build()); responseObserver.onCompleted(); } @Override public void onFailure(Throwable t) { responseObserver.onError(t);} }); } }
  80. 80. Google Cloud Platform Async Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ... Futures.addCallback(responsesFuture, new FutureCallback<List<Builder>>() { @Override public void onSuccess(@Nullable List<Builder> result) { responseObserver.onNext(responseBuilder.build()); responseObserver.onCompleted(); } @Override public void onFailure(Throwable t) { responseObserver.onError(t);} }); } }
  81. 81. Google Cloud Platform Panta rhei, "everything flows". Heraclitus ~2,400 years Before HTTP/1.x https://en.wikipedia.org/wiki/Heraclitus “ ”
  82. 82. Google Cloud Platform Service Definitions service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); } service TemperatureService { rpc GetCurrent(Coordinates) returns (Temperature); } service HumidityService { rpc GetCurrent(Coordinates) returns (HumidityResponse); } service WindService { rpc GetCurrent(Coordinates) returns (Wind); }
  83. 83. Google Cloud Platform Streaming Service Definitions service WeatherStreaming { rpc GetCurrent(WeatherRequest) returns (stream WeatherResponse); } service TemperatureServiceStreaming { rpc GetCurrent(Coordinates) returns (stream Temperature); } service HumidityServiceStreaming { rpc GetCurrent(Coordinates) returns (stream HumidityResponse); } service WindServiceStreaming { rpc GetCurrent(Coordinates) returns (stream Wind); }
  84. 84. Google Cloud Platform Bidirectional Streaming Service Definitions service WeatherStreaming { rpc GetCurrent(stream WeatherRequest) returns (stream WeatherResponse); } service TemperatureServiceStreaming { rpc GetCurrent(stream Coordinates) returns (stream Temperature); } service HumidityServiceStreaming { rpc GetCurrent(stream Coordinates) returns (stream HumidityResponse); } service WindServiceStreaming { rpc GetCurrent(stream Coordinates) returns (stream Wind); }
  85. 85. Google Cloud Platform Bidirectional Streaming Service Definitions service WeatherStreaming { rpc Observe(stream WeatherRequest) returns (stream WeatherResponse); } service TemperatureServiceStreaming { rpc Observe(stream Coordinates) returns (stream Temperature); } service HumidityServiceStreaming { rpc Observe(stream Coordinates) returns (stream HumidityResponse); } service WindServiceStreaming { rpc Observe(stream Coordinates) returns (stream Wind); }
  86. 86. Google Cloud Platform gRPC Service Definitions Unary RPCs where the client sends a single request to the server and gets a single response back, just like a normal function call. The client sends a request to the server and gets a stream to read a sequence of messages back. The client reads from the returned stream until there are no more messages. The client send a sequence of messages to the server using a provided stream. Once the client has finished writing the messages, it waits for the server to read them and return its response. Client streaming Both sides send a sequence of messages using a read-write stream. The two streams operate independently. The order of messages in each stream is preserved. BiDi streamingUnary Server streaming
  87. 87. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { private static final Logger logger = LogManager.getLogger(); private final TemperatureServiceStreamingStub temperatureService; private final HumidityServiceStreamingStub humidityService; private final WindServiceStreamingStub windService; public WeatherStreamingService(TemperatureServiceStreamingStub temperatureService, HumidityServiceStreamingStub humidityService, WindServiceStreamingStub windService) { this.temperatureService = temperatureService; this.humidityService = humidityService; this.windService = windService; } ...
  88. 88. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { ...
  89. 89. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { StreamObserver<Coordinates> temperatureClientStream = temperatureService.observe(newStreamObserver(responseObserver, (builder, temp) -> builder.setTemperature(temp))); StreamObserver<Coordinates> humidityClientStream = humidityService.observe(newStreamObserver(responseObserver, (builder, humidity) -> builder.setHumidity(humidity.getValue()))); StreamObserver<Coordinates> windClientStream = windService.observe(newStreamObserver(responseObserver, (builder, wind) -> builder.setWind(wind))); List<StreamObserver<Coordinates>> clientStreams = Arrays.asList(temperatureClientStream, humidityClientStream, windClientStream); ...
  90. 90. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { StreamObserver<Coordinates> temperatureClientStream = temperatureService.observe(newStreamObserver(responseObserver, (builder, temp) -> builder.setTemperature(temp))); StreamObserver<Coordinates> humidityClientStream = humidityService.observe(newStreamObserver(responseObserver, (builder, humidity) -> builder.setHumidity(humidity.getValue()))); StreamObserver<Coordinates> windClientStream = windService.observe(newStreamObserver(responseObserver, (builder, wind) -> builder.setWind(wind))); List<StreamObserver<Coordinates>> clientStreams = Arrays.asList(temperatureClientStream, humidityClientStream, windClientStream); ...
  91. 91. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { ... List<StreamObserver<Coordinates>> clientStreams = Arrays.asList(temperatureClientStream, humidityClientStream, windClientStream); ...
  92. 92. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { ... List<StreamObserver<Coordinates>> clientStreams = Arrays.asList(temperatureClientStream, humidityClientStream, windClientStream); return new StreamObserver<WeatherRequest>() { @Override public void onNext(WeatherRequest request) { clientStreams.forEach(s -> s.onNext(request.getCoordinates())); } @Override public void onError(Throwable t) { clientStreams.forEach(s -> s.onError(t)); } @Override public void onCompleted() { clientStreams.forEach(s -> s.onCompleted()); } }; } ...
  93. 93. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { ... List<StreamObserver<Coordinates>> clientStreams = Arrays.asList(temperatureClientStream, humidityClientStream, windClientStream); return new StreamObserver<WeatherRequest>() { @Override public void onNext(WeatherRequest request) { clientStreams.forEach(s -> s.onNext(request.getCoordinates())); } @Override public void onError(Throwable t) { clientStreams.forEach(s -> s.onError(t)); } @Override public void onCompleted() { clientStreams.forEach(s -> s.onCompleted()); } }; } ...
  94. 94. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { StreamObserver<Coordinates> temperatureClientStream = temperatureService.observe(newStreamObserver(responseObserver, (builder, temp) -> builder.setTemperature(temp))); StreamObserver<Coordinates> humidityClientStream = humidityService.observe(newStreamObserver(responseObserver, (builder, humidity) -> builder.setHumidity(humidity.getValue()))); StreamObserver<Coordinates> windClientStream = windService.observe(newStreamObserver(responseObserver, (builder, wind) -> builder.setWind(wind))); List<StreamObserver<Coordinates>> clientStreams = Arrays.asList(temperatureClientStream, humidityClientStream, windClientStream); ...
  95. 95. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { StreamObserver<Coordinates> temperatureClientStream = temperatureService.observe(newStreamObserver(responseObserver, (builder, temp) -> builder.setTemperature(temp))); ... } private <T> StreamObserver<T> newStreamObserver(StreamObserver<WeatherResponse> responseObserver, BiFunction<Builder, T, Builder> convert) { return new DelegatingStreamObserver<T, WeatherResponse>(responseObserver, t -> { return convert.apply(WeatherResponse.newBuilder(), t).build(); }); } ...
  96. 96. 96 Messaging applications. Games / multiplayer tournaments. Moving objects. Sport results. Stock market quotes. Smart home devices. You name it! BiDi Streaming Use-Cases
  97. 97. Google Cloud Platform gRPC Speaks Your Language ● Java ● Go ● C/C++ ● C# ● Node.js ● PHP ● Ruby ● Python ● Objective-C ● MacOS ● Linux ● Windows ● Android ● iOS Service definitions and client libraries Platforms supported
  98. 98. Google Cloud Platform Interoperability Java Service Python Service GoLang Service C++ Service gRPC Service gRPC Stub gRPC Stub gRPC Stub gRPC Stub gRPC Service gRPC Service gRPC Service gRPC Stub
  99. 99. Google Cloud Platform gRPC API in Google Cloud A massively scalable NoSQL database service. A fully-managed real-time messaging service. Speech to text conversion powered by machine learning. Cloud Speech APICloud Bigtable Cloud PubSub
  100. 100. Google Cloud Platform Some of the gRPC Adopters Your company could be here
  101. 101. Google Cloud Platform Netflix deprecated its RPC framework in favor of gRPC: “Our team has instead building an RPC solution on top of gRPC. We are doing this transition for two main reasons: multi-language support and better extensibility/composability through request interceptors. That’s our current plan moving forward.” https://github.com/Netflix/ribbon Some of the gRPC Adopters
  102. 102. Google Cloud Platform Transports: Client-side • Netty • OkHttp (Android) • InProcess Server-side • Netty • InProcess gRPC Java HTTP/2 RPC Client-Side App RPC Server-side App Service Definition (extends generated definition) ServerCall handler Transport (Netty) Channel Transport (Netty, OkHttp) Stub Future Stub Blocking Stub ServerCallClientCall
  103. 103. Google Cloud Platform Coverage & Simplicity The stack should be available on every popular development platform and easy for someone to build for their platform of choice. It should be viable on CPU & memory limited devices. gRPC Principles & Requirements http://www.grpc.io/blog/principles
  104. 104. Google Cloud Platform Tell Me Moar !!1
  105. 105. Google Cloud Platform Payload Agnostic
  106. 106. Google Cloud Platform Payload Agnostic - Thrift Support Example namespace java io.grpc.examples.thrift.helloworld struct HelloRequest { 1:string name } struct HelloResponse { 1:string message } service Greeter { HelloResponse sayHello(1:HelloRequest request); } https://github.com/grpc/grpc-java/tree/master/examples/thrift/src/main/java/io/grpc/examples/thrift/helloworld
  107. 107. 107 Fault Tolerance?
  108. 108. Google Cloud Platform Microservices The D'Artagnan [Our] Service Not D'Artagnan Service Not D'Artagnan Service Not D'Artagnan Service Not D'Artagnan Service Not D'Artagnan Service Not D'Artagnan Service Not D'Artagnan Service
  109. 109. 109 Things will break. Don’t pretend you can eliminate every possible source of failure. Integration points are #1 killer of software systems. Fault Tolerance?
  110. 110. Google Cloud Platform Use time-outs! Captain “ ”* Original photo taken by Alexandr Gusew.
  111. 111. Google Cloud Platform Use time-outs! Because your code cannot wait forever. Captain “ ”* Original photo taken by Alexandr Gusew.
  112. 112. Google Cloud Platform Timeouts? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms ? ? ? ? ? ? ? ? ?
  113. 113. Google Cloud Platform Default Timeout For Every Service? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms
  114. 114. Google Cloud Platform Default Timeout For Every Service? Gateway 20 ms 10 ms 10 ms FastFastFast 30 ms 40 ms 20 ms 130 ms 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout
  115. 115. Google Cloud Platform Default Timeout For Every Service? Gateway 20 ms 30 ms 10 ms 130 ms 100 ms 20 ms FastFairFair 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout
  116. 116. Google Cloud Platform Default Timeout For Every Service? 20 ms 30 ms 10 ms 130 ms 100 ms 20 ms 310 ms FastFairFair 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout
  117. 117. Google Cloud Platform Default Timeout For Every Service? Gateway 30 ms 80 ms 100 ms SlowFair 210 ms 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout
  118. 118. Google Cloud Platform Default Timeout For Every Service? 30 ms 80 ms 100 ms SlowFair 210 ms 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout
  119. 119. Google Cloud Platform Default Timeout For Every Service? 80 ms 100 ms SlowFair 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout Still working!
  120. 120. Google Cloud Platform Default Timeout For Every Service? 80 ms 100 ms 90 ms SlowSlowFair 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout
  121. 121. Google Cloud Platform Default Timeout For Every Service? 80 ms 100 ms 90 ms SlowSlowFair 270 ms 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout
  122. 122. Google Cloud Platform Default Timeout For Every Service? 200 ms timeout 200 ms timeout 200 ms timeout 80 ms 100 ms 90 ms SlowSlow 270 ms 200 ms timeout
  123. 123. Google Cloud Platform Tune Timeout For Every Service? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms 190 ms 190 ms 190 ms 110 ms 50 ms 110 ms 50 ms 110 ms 50 ms
  124. 124. Google Cloud Platform Tune Timeout For Every Service? Gateway 200 ms timeout 190 ms timeout 110 ms timeout 30 ms 80 ms 25 ms FastFair 55 ms Fast 30 ms 50 ms timeout
  125. 125. Google Cloud Platform Tune Timeout For Every Service? Gateway 200 ms timeout 190 ms timeout 110 ms timeout 30 ms 80 ms 25 ms Fair 55 ms Fast 30 ms 50 ms timeout
  126. 126. Google Cloud Platform Timeouts? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms ? ? ? ? ? ? ? ? ?
  127. 127. Google Cloud Platform Timeouts for Real World? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms
  128. 128. Google Cloud Platform Timeout Propagation Gateway 90 ms timeout = 200 - 40 ms timeout = 160 - 90 - 20 ms timeout = 200 ms 40 ms 20 ms 20 ms 60 ms timeout reached! 230 ms
  129. 129. 129 Timeouts in gRPC
  130. 130. 130 gRPC Java does not support timeouts. Timeouts in gRPC
  131. 131. 131 gRPC Java does not support timeouts. gRPC supports deadlines instead! gRPC Deadlines WeatherResponse response = client.withDeadlineAfter(200, MILLISECONDS) .getCurrent(request);
  132. 132. 132 First-class feature in gRPC. Deadline is an absolute point in time. Deadline indicates to the server how long the client is willing to wait for an answer. RPC will fail with DEADLINE_EXCEEDED status code when deadline reached. gRPC Deadlines
  133. 133. Google Cloud Platform gRPC Deadline Propagation Gateway 90 ms Now = 1476600000000 Deadline = 1476600000200 40 ms 20 ms 20 ms 60 ms withDeadlineAfter(200, MILLISECONDS) Now = 1476600000040 Deadline = 1476600000200 Now = 1476600000150 Deadline = 1476600000200 Now = 1476600000230 Deadline = 1476600000200 DEADLINE_EXCEEDED DEADLINE_EXCEEDED DEADLINE_EXCEEDED DEADLINE_EXCEEDED
  134. 134. 134 Deadlines are automatically propagated! Can be accessed by the receiver! gRPC Deadlines Context context = Context.current(); context.getDeadline().isExpired(); context.getDeadline() .timeRemaining(MILLISECONDS); context.getDeadline().runOnExpiration(() -> logger.info("Deadline exceeded!"), exec);
  135. 135. 135 Deadlines are expected. What about unpredictable cancellations? • User cancelled request. • Caller is not interested in the result any more. • etc Cancellation?
  136. 136. Google Cloud Platform Cancellation? GW Busy Busy Busy Busy Busy Busy Busy Busy Busy RPC Active RPC Active RPC Active RPC Active RPC Active RPCActive RPC Active RPC Active RPC Active RPC
  137. 137. Google Cloud Platform Cancellation? GW Busy Busy Busy Busy Busy Busy Busy Busy Busy Active RPC Active RPC Active RPC Active RPC Active RPCActive RPC Active RPC Active RPC Active RPC
  138. 138. Google Cloud Platform Cancellation Propagation GW Idle Idle Idle Idle Idle Idle Idle Idle Idle
  139. 139. 139 Automatically propagated. RPC fails with CANCELLED status code. Cancellation status be accessed by the receiver. Server (receiver) always knows if RPC is valid! gRPC Cancellation
  140. 140. 140 Automatically propagated. RPC fails with CANCELLED status code. Cancellation status be accessed by the receiver. Server (receiver) always knows if RPC is valid! gRPC Cancellation
  141. 141. Google Cloud Platform gRPC Cancellation public class WeatherService extends WeatherGrpc.WeatherImplBase { ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ServerCallStreamObserver<WeatherResponse> streamObserver = (ServerCallStreamObserver<WeatherResponse>) responseObserver; streamObserver.isCancelled(); ...
  142. 142. Google Cloud Platform gRPC Cancellation public class WeatherService extends WeatherGrpc.WeatherImplBase { ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ServerCallStreamObserver<WeatherResponse> streamObserver = (ServerCallStreamObserver<WeatherResponse>) responseObserver; streamObserver.setOnCancelHandler(() -> { cleanupCallResources(); logger.info("Call cancelled by client!"); }); ...
  143. 143. Google Cloud Platform gRPC Context public class WeatherService extends WeatherGrpc.WeatherImplBase { ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Context.current().getDeadline(); Context.current().isCancelled(); Context.current().cancellationCause(); ...
  144. 144. Google Cloud Platform gRPC Context public class WeatherService extends WeatherGrpc.WeatherImplBase { ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Context.current().addListener(context -> { cleanupCallResources(); logger.info("Call cancelled by client!"); }, executor) ...
  145. 145. Google Cloud Platform More Control?
  146. 146. Google Cloud Platform BiDi Streaming - Slow Client Fast Server Request Responses Slow Client CANCELLED UNAVAILABLE RESOURCE_EXHAUSTED
  147. 147. Google Cloud Platform BiDi Streaming - Slow Server Slow Server Request Response Fast Client CANCELLED UNAVAILABLE RESOURCE_EXHAUSTED Requests
  148. 148. Google Cloud Platform Client-Side Flow-Control Server Request, count = 4 Client 2 Responses 4 Responses Count = 2
  149. 149. Google Cloud Platform Server-Side Flow-Control Server Request Client Response count = 2 2 Requests count = 3 3 Requests
  150. 150. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver requestStream) { requestStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3);
  151. 151. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver requestStream) { requestStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3);
  152. 152. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver requestStream) { requestStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3);
  153. 153. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver requestStream) { requestStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3);
  154. 154. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver requestStream) { requestStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3);
  155. 155. Google Cloud Platform Flow-Control (Client-Side) public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { ServerCallStreamObserver<WeatherResponse> streamObserver = (ServerCallStreamObserver<WeatherResponse>) responseObserver; streamObserver.setOnReadyHandler(() -> { while (streamObserver.isReady()) { streamObserver.onNext(calculateWeather()); } }); ...
  156. 156. 156 Flow-control helps to balance computing power and network capacity between client and server. gRPC supports both client- and server-side flow control. Disabled by default. Flow-Control Photo taken by Andrey Borisenko.
  157. 157. Google Cloud Platform Interceptors Client Server Request Response Client interceptors Server interceptors
  158. 158. Google Cloud Platform Client Interceptor public class HeaderClientInterceptor implements ClientInterceptor { @Override public <ReqT, RespT> ClientCall<ReqT, RespT> interceptCall(MethodDescriptor<ReqT, RespT> method, CallOptions callOptions, Channel next) { } }
  159. 159. Google Cloud Platform Client Interceptor public class HeaderClientInterceptor implements ClientInterceptor { private static Metadata.Key<String> customHeadKey = Metadata.Key.of("custom_header_key", ASCII_STRING_MARSHALLER); @Override public <ReqT, RespT> ClientCall<ReqT, RespT> interceptCall(MethodDescriptor<ReqT, RespT> method, CallOptions callOptions, Channel next) { return new SimpleForwardingClientCall<ReqT, RespT>(next.newCall(method, callOptions)) { @Override public void start(Listener<RespT> responseListener, Metadata headers) { // put custom header headers.put(customHeadKey, "customRequestValue"); super.start(responseListener, headers); } }; } }
  160. 160. Google Cloud Platform Server Interceptor public class HeaderServerInterceptor implements ServerInterceptor { @Override public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(ServerCall<ReqT, RespT> call, Metadata requestHeaders, ServerCallHandler<ReqT, RespT> next) { } }
  161. 161. Google Cloud Platform Server Interceptor public class HeaderServerInterceptor implements ServerInterceptor { private static final Logger logger = LogManager.getLogger(); private static Metadata.Key<String> customHeadKey = Metadata.Key.of("custom_header_key", ASCII_STRING_MARSHALLER); @Override public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(ServerCall<ReqT, RespT> call, Metadata requestHeaders, ServerCallHandler<ReqT, RespT> next) { String customHeaderValue = requestHeaders.get(customHeadKey); logger.info("header received from client: {}", customHeaderValue); return next.startCall(call, requestHeaders); } }
  162. 162. Google Cloud Platform Server Interceptor public class HeaderServerInterceptor implements ServerInterceptor { private static final Logger logger = LogManager.getLogger(); private static Metadata.Key<String> customHeadKey = Metadata.Key.of("custom_header_key", ASCII_STRING_MARSHALLER); @Override public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(ServerCall<ReqT, RespT> call, Metadata requestHeaders, ServerCallHandler<ReqT, RespT> next) { String customHeaderValue = requestHeaders.get(customHeadKey); logger.info("header received from client: {}", customHeaderValue); return next.startCall(call, requestHeaders); } }
  163. 163. Google Cloud Platform Metadata Exchange Metadata is first-class feature in gRPC. Lets the client provide information associated with the call to the server and vice versa in the form of a list of key-value pairs. Keys are strings; values can be either strings or binary data.
  164. 164. Google Cloud Platform Metadata Exchange Metadata is first-class feature in gRPC. Lets the client provide information associated with the call to the server and vice versa in the form of a list of key-value pairs. Keys are strings; values can be either strings or binary data. Can be sent in headers or trailers. http://www.grpc.io/docs/guides/concepts.html#metadata
  165. 165. Google Cloud Platform Service Discovery & Load Balancing HTTP/2 RPC Client-Side App RPC Server-side App Service Definition (extends generated definition) ServerCall handler Transport Channel Transport Stub Future Stub Blocking Stub ServerCallClientCall
  166. 166. Google Cloud Platform Service Discovery & Load Balancing HTTP/2 RPC Client-Side App Channel Transport Stub Future Stub Blocking Stub ClientCall Instance #1 Instance #N Instance #2 RPC Server-side Apps
  167. 167. Google Cloud Platform Service Discovery & Load Balancing HTTP/2 RPC Client-Side App Channel Stub Future Stub Blocking Stub ClientCall Instance #1 Instance #1 Instance #N Instance #2 RPC Server-side Apps Tran #1 Tran #2 Tran #N
  168. 168. Google Cloud Platform Service Discovery & Load Balancing Channel is a logical connection to a service, which is a collection of servers. NameResolver is a pluggable component that resolves a service name and return addresses to the caller. It has no knowledge of load-balancing. There will be different NameResolvers to support different name-systems, e.g., DNS, ZooKeeper etc. LoadBalancer is a pluggable component that receives resolved addresses from NameResolver and selects a Transport for Channel when asked.
  169. 169. Google Cloud Platform Service Discovery & Load Balancing ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv") .nameResolverFactory(new DnsNameResolverProvider()) .loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance()) .build();
  170. 170. Google Cloud Platform Service Discovery & Load Balancing ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv") .nameResolverFactory(new DnsNameResolverProvider()) .loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance()) .build();
  171. 171. Google Cloud Platform Service Discovery & Load Balancing ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv") .nameResolverFactory(new DnsNameResolverProvider()) .loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance()) .build(); Design documents: • Load Balancing in gRPC: https://github.com/grpc/grpc/blob/master/doc/load-balancing.md • gRPC Java Name Resolution and Load Balancing: http://tiny.cc/grpc-java-lb
  172. 172. Google Cloud Platform Layered Architecture HTTP/2 RPC Client-Side App Channel Stub Future Stub Blocking Stub ClientCall RPC Server-side Apps Tran #1 Tran #2 Tran #N Service Definition (extends generated definition) ServerCall handler Transport ServerCall NameResolver LoadBalancer Pluggable Load Balancing and Service Discovery
  173. 173. Google Cloud Platform gRPC Principles & Requirements Layered Key facets of the stack must be able to evolve independently. A revision to the wire-format should not disrupt application layer bindings. http://www.grpc.io/blog/principles
  174. 174. Google Cloud Platform Testing Support In-process server transport: fully-featured, high performance, and useful in testing. In-process client side channel: fully-featured, high performance, and useful in testing.
  175. 175. Google Cloud Platform Testing Support - InProcess WeatherServiceAsync weatherService = new WeatherServiceAsync(tempService, humidityService, windService); Server grpcServer = InProcessServerBuilder.forName("weather") .addService(weatherService).build();
  176. 176. Google Cloud Platform Testing Support - InProcess WeatherServiceAsync weatherService = new WeatherServiceAsync(tempService, humidityService, windService); Server grpcServer = InProcessServerBuilder.forName("weather") .addService(ServerInterceptors.intercept(weatherService, interceptor)).build();
  177. 177. Google Cloud Platform Testing Support - InProcess WeatherServiceAsync weatherService = new WeatherServiceAsync(tempService, humidityService, windService); Server grpcServer = InProcessServerBuilder.forName("weather") .addService(ServerInterceptors.intercept(weatherService, interceptor)).build(); Channel grpcChannel = InProcessChannelBuilder.forName("weather").build(); WeatherBlockingStub stub = WeatherGrpc.newBlockingStub(grpcChannel).withDeadlineAfter(100, MILLISECONDS);
  178. 178. Google Cloud Platform Testing Support - InProcess WeatherBlockingStub stub = WeatherGrpc.newBlockingStub(grpcChannel).withDeadlineAfter(100, MILLISECONDS); @Test public void returnWeatherResponseWhenAllDependentServicesReply() { WeatherRequest request = WeatherRequest.newBuilder().setCoordinates( Coordinates.newBuilder().setLatitude(420000000).setLongitude(-720000000)).build(); WeatherResponse result = stub.getCurrent(request); ...
  179. 179. Google Cloud Platform Testing Support In-process server transport: fully-featured, high performance, and useful in testing. In-process client side channel: fully-featured, high performance, and useful in testing. MetadataUtils for testing metadata headers and trailer.
  180. 180. Google Cloud Platform Testing Support - Metadata @Test public void sendCustomHeaderMetaData() { AtomicReference<Metadata> headers = new AtomicReference<>(); AtomicReference<Metadata> trailers = new AtomicReference<>(); WeatherBlockingStub metaCapturingStub = MetadataUtils.captureMetadata(stub, headers, trailers); metaCapturingStub.getCurrent(request); Metadata headersMeta = headers.get(); assertThat(headersMeta.get(customHeadKey)).isEqualTo(expectedCustomHeadValue); // same can be done for trailers }
  181. 181. Google Cloud Platform Testing Support In-process server transport: fully-featured, high performance, and useful in testing. In-process client side channel: fully-featured, high performance, and useful in testing. MetadataUtils for testing metadata headers and trailer. grpc-testing - additional test utility functions useful for writing unit and integration tests: https://github.com/grpc/grpc-java/tree/master/testing
  182. 182. Google Cloud Platform Distributed Tracing A B C D E F G H J
  183. 183. Google Cloud Platform Distributed Tracing A B C D E F G H J
  184. 184. Google Cloud Platform Distributed Tracing Paper: Dapper, a Large-Scale Distributed Systems Tracing Infrastructure: http://research.google.com/pubs/pub36356.html Implementation: Zipkin is a distributed tracing system: zipkin.io (JVM, Go, C#, Python, Ruby). Brave is a pure-Java distributed tracing implementation compatible with Zipkin (no Scala required): https://github.com/openzipkin/brave Brave-gRPC integrates Braves with gRPC.
  185. 185. Google Cloud Platform Distributed Tracing with Zipkin/Brave // server-side Server grpcServer = NettyServerBuilder.forPort(8090) .addService(ServerInterceptors.intercept(new WeatherService(), new BraveGrpcServerInterceptor(brave("wind")))) .build();
  186. 186. Google Cloud Platform Distributed Tracing with Zipkin/Brave // server-side Server grpcServer = NettyServerBuilder.forPort(8090) .addService(ServerInterceptors.intercept(new WeatherService(), new BraveGrpcServerInterceptor(brave("wind")))) .build(); // client-side Channel windChannel = NettyChannelBuilder.forAddress(windserverAddress()).build(); WindServiceStub windClient = WindServiceGrpc.newStub( ClientInterceptors.intercept(windChannel, new BraveGrpcClientInterceptor(brave("weather_to_wind"))));
  187. 187. Google Cloud Platform Distributed Tracing with Zipkin/Brave
  188. 188. Google Cloud Platform Distributed Tracing with Zipkin/Brave
  189. 189. Google Cloud Platform Distributed Tracing with Zipkin/Brave
  190. 190. Google Cloud Platform Distributed Tracing with Zipkin/Brave Weather Weather Wind Conversion Units Temp Humidity
  191. 191. Google Cloud Platform Distributed Tracing with Zipkin/Brave
  192. 192. 192 gRPC Gateway Photo taken by Andrey Borisenko.
  193. 193. 193 Migration. Testing. Swagger / OpenAPI tooling. gRPC Gateway Photo taken by Andrey Borisenko. https://github.com/grpc-ecosystem/grpc-gateway
  194. 194. Google Cloud Platform More Features Monitoring: • gRPC Prometheus. • More Monitoring APIs are coming. Built-in authentication mechanisms: • SSL/TLS. • Token-based authentication with Google. • Authentication API. Payload compression: https://github.com/grpc/grpc/blob/master/doc/compression.md
  195. 195. Google Cloud Platform Growing Community and Ecosystem https://github.com/grpc-ecosystem Polyglot is a universal grpc command line client. grpc-gateway generates a reverse-proxy server which translates a RESTful JSON API into gRPC. OpenTracing is a set of consistent, expressive, vendor-neutral APIs for distributed tracing and context propagation. Prometheus monitoring support for grpc-java and grpc-go.
  196. 196. Google Cloud Platform Pluggable Large distributed systems need security, health-checking, load-balancing and failover, monitoring, tracing, logging, and so on. Implementations should provide extensions points to allow for plugging in these features and, where useful, default implementations. gRPC Principles & Requirements http://www.grpc.io/blog/principles
  197. 197. Google Cloud Platform Try It Out! http://grpc.io gRPC on Github: https://github.com/grpc gRPC Java on Github: https://github.com/grpc/grpc-java Google group: grpc-io@googlegroups.com gRPC Java quickstart: http://www.grpc.io/docs/quickstart/java.html gRPC Java tutorial: http://www.grpc.io/docs/tutorials/basic/java.html gRPC contribution: https://github.com/grpc/grpc-contrib
  198. 198. Google Cloud Platform Takeaways HTTP/2 is a high performance production-ready multiplexed bidirectional protocol. gRPC (http://grpc.io): • HTTP/2 transport based, open source, general purpose standards-based, feature-rich RPC framework. • Bidirectional streaming over one single TCP connection. • Netty transport provides asynchronous and non-blocking I/O. • Deadline and cancellations propagation. • Client- and server-side flow-control. • Layered, pluggable and extensible. • Supports 10 programming languages. • Build-in testing support. • Production-ready (current version is 1.0.1) and growing ecosystem.
  199. 199. Google Cloud Platform Questions?
  200. 200. Thank You
  201. 201. Google Cloud Platform Images Used Special thanks for the provided photos: • Andrey Borisenko • Alexandr Gusew Photo from https://www.google.com/about/datacenters page: • https://www.google.com/about/datacenters/gallery/#/tech/12 Photos licenced by Creative Commons 2.0 https://creativecommons.org/licenses/by/2.0/ : • https://www.flickr.com/photos/13800911@N08/3557747851/ by DirectDish • https://www.flickr.com/photos/marcalandavis/45657810/ by Marc Davis • https://www.flickr.com/photos/garryknight/5754661212/ by Garry Knight • https://www.flickr.com/photos/sodaigomi/21128888345/ by sodai gomi • https://www.flickr.com/photos/zengame/15972170944/ by Zengame • https://www.flickr.com/photos/londonmatt/18496249193/ by Matt Brown • https://www.flickr.com/photos/gazeronly/8058105980/ by torbakhopper U.S. Government Works: • https://www.flickr.com/photos/cbpphotos/8653133856/ by Josh Denmark Free of known restrictions: https://commons.wikimedia.org/wiki/File:He_can%27t_fix_guns_in_the_air%5E_Build_%60em_right%5E_Keep_%60em_firing%5E_-_NARA_-_535050.jpg

×