Successfully reported this slideshow.
Your SlideShare is downloading. ×

Cloud native IPC for Microservices Workshop @ Containerdays 2022

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 30 Ad

Cloud native IPC for Microservices Workshop @ Containerdays 2022

Download to read offline

Container Days Hamburg, September 2022, Dirk Kröhan, (Software Architect), Andreas Zitzelsberger (@andreasz82, Lead Software Architect) at QAware.
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==

This workshop focuses on modern and efficient Inter Process Communication (IPC) for microservices. We start with a REST API, built using JAX-RS and Quarkus to briefly discuss the pros and cons of this approach. Then, we will extend the API with an efficient Protobuf payload representation in order to finally transform the API into a fully fledged high-performance gRPC interface definition. But that’s not all! To put some extra icing on the cake, this workshop will demonstrate how to consume the gRPC service from a JavaScript web client and also how to completely generate a matching REST API from an enhanced gRPC interface definition to ensure full interoperability in a microservice architecture.

Container Days Hamburg, September 2022, Dirk Kröhan, (Software Architect), Andreas Zitzelsberger (@andreasz82, Lead Software Architect) at QAware.
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==

This workshop focuses on modern and efficient Inter Process Communication (IPC) for microservices. We start with a REST API, built using JAX-RS and Quarkus to briefly discuss the pros and cons of this approach. Then, we will extend the API with an efficient Protobuf payload representation in order to finally transform the API into a fully fledged high-performance gRPC interface definition. But that’s not all! To put some extra icing on the cake, this workshop will demonstrate how to consume the gRPC service from a JavaScript web client and also how to completely generate a matching REST API from an enhanced gRPC interface definition to ensure full interoperability in a microservice architecture.

Advertisement
Advertisement

More Related Content

More from QAware GmbH (20)

Advertisement

Cloud native IPC for Microservices Workshop @ Containerdays 2022

  1. 1. qaware.de Cloud native IPC for Microservices Containerdays 2022
  2. 2. QAware | 2 Andreas Zitzelsberger Lead Software Architect @andreasz82 #cloudnative #qaware #gernperDude #🍺 Dirk Kröhan Software Architect #qaware #mainz #cloudnative ⚑
  3. 3. Exercise 0: Tools & infrastructure Java Development Environment: ■ IDE of your choice. Recommendations: ■ Visual Studio Code: https://code.visualstudio.com ■ IntelliJ IDEA: https://www.jetbrains.com/de-de/idea/download ■ OpenJDK 17: https://adoptium.net/de/temurin/releases ■ Node.js >= 16.x: https://nodejs.org/en/download/ Additional Tooling: ■ Go: https://go.dev/doc/install ■ Protobuf: https://grpc.io/docs/protoc-installation/ ■ Buf CLI: https://docs.buf.build/installation ■ Tilt: https://docs.tilt.dev/install.html ■ [Alternative] Skaffold: https://skaffold.dev ■ [Optional] Bash Shell and GNU Make QAware | 3 Local Docker & Kubernetes Installation: ■ Docker Desktop: https://www.docker.com/products/docker-desktop ■ [Alternative] Rancher Desktop: https://rancherdesktop.io ■ [Alternative] Minikube + kubectl + Docker CLI: https://kubernetes.io/docs/tasks/tools/ ⚠ Breaking bug in protobuf 21.1. Downgrade to 20.1. Mac/Homebrew: brew install protobuf@3 brew link --overwrite protobuf@3
  4. 4. Agenda QAware | 4 From To Duration Topic 09:00 09:30 00:30 Introduction 09:30 09:45 00:15 Exercise 0: Tools & infrastructure 09:45 10:30 00:45 Exercise 1: Protobuf with Quarkus and JAX-RS 10:30 10:45 00:15 Break 10:45 11:30 00:45 Exercise 2: Quarkus with a gRPC API 11:30 12:15 00:45 Exercise 3: gRPC REST Gateway 12:15 12:45 00:30 Exercise 4: gRPC web client with Envoy 12:45 13:00 00:15 Wrap-Up
  5. 5. Our Exercises QAware | 5 QAware | 5 QAware | 5 REST Beer Service REST Beer Service application/json application/x-protobuf gRPC Beer Service gRPC Beer Service gRPC Beer Web UI gRPC Beer Client gRPC REST Gateway application/json gRPC LB Nginx gRPC gRPC gRPC gRPC gRPC Web UI gRPC Web Envoy TypeScript
  6. 6. Choose your framework and language Results https://www.menti.com/xpuuv2qakr
  7. 7. qaware/from-rest-to-grpc-workshop
  8. 8. “One cannot not communicate.” - Paul Watzlawick
  9. 9. RPC is the most common API style QAware | 9 Style Examples Comments Remote Procedure Calls (RPC) GRPC, JSON-over-HTTP, CORBA Classic request-response pattern. A request is expected to trigger an action that may yield a result. Resource Oriented REST/JSON Resources are inspected / modified via CRUD methods (HTTP verbs). Data Oriented GraphQL, SQL Data structures are presented in the API and can be queried and modified as needed. Message Oriented Kafka, JMS, MQTT Messages are passed with defined delivery semantics.
  10. 10. A Quick History Lesson on Inter Process Communication (IPC) QAware | 10 DCOM 18.09.1996 Win95 RPC 14.01.1976 RFC 707 REST 2000 by Roy T. Fielding Java RMI Feb 1997 JDK 1.1 HTTP/1.0 Mai 1996 RFC 1945 HTTP/1.1 Juni 1999 RFC 2616 HTTP/2.0 Mai 2015 RFC 7540 SOAP 1.2 2003 RPC Oct 1983 Birrel und Nielson CORBA 1.0 Oct 1991 CORBA 2.0 August 1996 CORBA 2.3 Juni 1999 XML-RPC 1998 gRPC 1.0 Aug 2016 RESTful Applications 2014 (?) CORBA 3.0 July 2002
  11. 11. Exercise 1 QAware | 11 QAware | 11 QAware | 11 REST Beer Service REST Beer Service application/json application/x-protobuf gRPC Beer Service gRPC Beer Service gRPC Beer Web UI gRPC Beer Client gRPC REST Gateway application/json gRPC LB Nginx gRPC gRPC gRPC gRPC gRPC Web UI gRPC Web Envoy TypeScript
  12. 12. Wifi Hotspots QAware | 12 Hotspot 1: SSID: grpc-workshop Password: grpc-rules Hotspot 2 SSID: "iPhone von Dirk" Password: grpcworkshop
  13. 13. HTTP/1.1 200 OK Content-Length: 139 Content-Type: application/json; charset=utf-8 Date: Wed, 10 Nov 2021 10:21:54 GMT { "alcohol": 5.6, "asin": "B01AU6LWNC", "brand": "Augustiner Brauerei München", "country": "Germany", "name": "Edelstoff Exportbier", "type": "Lager" } GET /api/beers/B01AU6LWNC HTTP/1.1 Accept: application/json Accept-Encoding: gzip, deflate Connection: keep-alive Host: localhost:8080 User-Agent: HTTPie/2.5.0 REST APIs GET /api/beers POST /api/beers GET /api/beers/{asin} PUT /api/beers/{asin} DELETE /api/beers/{asin}
  14. 14. Richardson REST Maturity Model QAware | 14 https://martinfowler.com/articles/richardsonMaturityModel.html POST /bookingService HTTP/1.1 [various other headers] <makeBookingRequest date="2010-01-04" persons="2"/> POST /bookings HTTP/1.1 [various other headers] <getBookingRequest id="ID-1234567890" user"lreimer"/> GET /bookings/1234567890?user=lreimer HTTP/1.1 Accept: application/json [various other headers] GET /bookings/1234567890?user=lreimer HTTP/1.1 Accept: application/json Link: /users/lreimer [various other headers]
  15. 15. Nobody does REST… QAware | 15 ■ The purpose why we build REST-like APIs: We do not want to explain our API to each API client – REST is a “standard” a lot of developers already are familiar with or know about ■ It is all about conventions and discussions about what is conventions – What is a resource? Singular vs. Plural? API Versioning? Bulk APIs? Patch or not to patch? ■ REST-like APIs are great if you do not know your API clients (e.g. a public API) – Resource oriented with a good portion of “standards” and conventions ■ For APIs with known clients RPC is the easier path – Having a collection of functions that operate over the Internet is often enough
  16. 16. QAware | 16 1. The network is reliable 2. Latency is zero 3. Bandwidth is infinite 4. The network is secure 5. Topology doesn’t change 6. There is one administrator 7. Transport cost is zero 8. The network is homogeneous The 8 Fallacies of Distributed Computing
  17. 17. Protocol Buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. QAware | 17 ■ https://developers.google.com/protocol-buffers ■ Like XML or JSON - just smaller, faster and easier! ■ Google Protobuf uses an efficient binary format to serialize data structures. ■ An Interface Definition Language (IDL) is used to define data structures and message payloads. Many primitive types, enums, maps, arrays, nested types. ■ Protocol Buffers supports code generation for Java, Python, Objective-C, C++, Kotlin, Dart, Go, Ruby und C#. ■ Protobuf supports evolution as well as extension of schemas. Backwards and forwards compatibility are supported: – you must not change the tag numbers of any existing fields. – you may delete fields. – you may add new fields but you must use fresh tag numbers (i.e. tag numbers that were never used in this protocol buffer, not even by deleted fields).
  18. 18. syntax = "proto3"; option java_package = "hands.on.grpc"; option java_outer_classname = "BeerProtos"; package beer; message Beer { string asin = 1; string name = 2; string brand = 3; string country = 4; float alcohol = 5; enum BeerType{ IndianPaleAle = 0; SessionIpa = 1; Lager = 2; } BeerType type = 6; } // Protobuf marshalling protoBeer = BeerProtos.Beer.newBuilder() .setAsin("B079V9ZDNY") .setName("Drunken Sailor") .build(); byte[] bytes = protoBeer.toByteArray(); // Protobuf unmarshalling protoBeer = BeerProtos.Beer.parseFrom(bytes); $ ./gradlew generateProto $ protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/beer.proto
  19. 19. JSON vs Protobuf Performance QAware | 19 ■ Protobuf on a non-compressed environment, the requests took 78% less time than the JSON requests. The binary format performed almost 5 times faster than the text format. ■ Protobuf requests on a compressed environment, the difference was even bigger. Protobuf performed 6 times faster, taking only 25ms to handle requests that took 150ms on a JSON format. https://auth0.com/blog/beating-json-performance-with-protobuf/ https://blog.qaware.de/posts/binary-data-format-comparison/ Disclaimer: please perform your own benchmarks for your specific use case!
  20. 20. Protobuf is well suited for high-volume use cases with known interface partners QAware | 20 Protobuf (gRPC) JSON JSON w. gzip Message Footprint Small Large Small Serialization Time Medium Slow Even Slower Out-of-Band Processing Yes No No Observability Limited Good Good Wideness of Usage Good Ubiquitous Ubiquitous Use When High volume Known interface partners Internal use Low latency requirement Low Volume Unknown interface partners External use Relaxed latency requirement
  21. 21. Exercise 2 QAware | 21 QAware | 21 QAware | 21 REST Beer Service REST Beer Service application/json application/x-protobuf gRPC Beer Service gRPC Beer Service gRPC Beer Web UI gRPC Beer Client gRPC REST Gateway application/json gRPC LB Nginx gRPC gRPC gRPC gRPC Web UI gRPC Web Envoy TypeScript gRPC gRPC
  22. 22. The Genesis of gRPC (gRPC Remote Procedure Calls) QAware | 22 ■ Origin: A Google-internal RPC framework called Stubby, created ca. 2001 – Not based on any standard – Tightly coupled to Google's internal infrastructure – Considered unsuitable for public release ■ Improved and open sourced in 2016 as gRPC – Trigger: Advent of HTTP/2 which covers many of Stubbys features – Main design goals: • Low latency • Low bandwidth usage • HTTP transport ■ gRPC became a CNCF incubating project in 2017
  23. 23. gRPC. A modern, high performance, open source and universal RPC framework. ■ Uses HTTP/2 as modern Web-friendly transport protocol (Multiplexing, TLS, compression, …) ■ Supports several types of communication: classic request-response as well as streaming from Client-side, Server-side, Uni- and Bi-Directional ■ Uses Protocol Buffers as efficient binary payload format – gRPC is encoding agnostic – Other encoders support: JSON, Thrift, Avro, Flatbuffers, Cap’n Proto, and even raw bytes ■ Support various load balancing options: proxy, client-side and look-aside balancing ■ Flexible support for tracing, health checks and authentication ■ Client and server code can be generated from the IDL easily for several languages – https://github.com/grpc/grpc-go – https://buf.build ■ gRPC is a CNCF incubating project QAware | 23
  24. 24. syntax = "proto3"; option java_package = "hands.on.grpc"; option java_outer_classname = "BeerProtos"; import "google/protobuf/empty.proto"; package beer; service BeerService { rpc AllBeers (google.protobuf.Empty) returns (GetBeersResponse) {} rpc GetBeer (GetBeerRequest) returns (GetBeerResponse) {} rpc CreateBeer (CreateBeerRequest) returns (google.protobuf.Empty) {} rpc UpdateBeer (UpdateBeerRequest) returns (google.protobuf.Empty) {} rpc DeleteBeer (DeleteBeerRequest) returns (google.protobuf.Empty) {} } // more Protobuf message definitions ...
  25. 25. Exercise 3 QAware | 25 QAware | 25 QAware | 25 REST Beer Service REST Beer Service application/json application/x-protobuf gRPC Beer Service gRPC Beer Service gRPC Beer Web UI gRPC Beer Client gRPC REST Gateway application/json gRPC LB Nginx gRPC gRPC gRPC gRPC Web UI gRPC Web Envoy TypeScript gRPC gRPC
  26. 26. Exercise 4 QAware | 26 QAware | 26 QAware | 26 REST Beer Service REST Beer Service application/json application/x-protobuf gRPC Beer Service gRPC Beer Service gRPC Beer Web UI gRPC Beer Client gRPC REST Gateway application/json gRPC LB Nginx gRPC gRPC gRPC gRPC gRPC Web UI gRPC Web Envoy TypeScript gRPC
  27. 27. The gRPC ecosystem in a nutshell. ■ Different projects from the gRPC ecosystem enable good interoperability – gRPC Gateway https://grpc-ecosystem.github.io/grpc-gateway/ – gRPC Web https://grpc.io/docs/platforms/web/quickstart/ ■ Support various load balancing options: proxy, client-side and look-aside balancing – Nginx https://nginx.org/en/docs/http/ngx_http_grpc_module.html – Envoy https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_protocols/grpc QAware | 27
  28. 28. import "google/protobuf/empty.proto"; import "google/api/annotations.proto"; service BeerService { rpc AllBeers (google.protobuf.Empty) returns (GetBeersResponse) { option (google.api.http) = { get: "/api/beers" }; } rpc GetBeer (GetBeerRequest) returns (GetBeerResponse) { option (google.api.http) = { get: "/api/beers/{asin}" response_body: "beer" }; } rpc CreateBeer (CreateBeerRequest) returns (google.protobuf.Empty) { option (google.api.http) = { post: "/api/beers" body: "*" }; } // more definitions … } Map gRPC call to GET request path Map {asin} path param to request Use beer field as response body Map POST body to request
  29. 29. REST on the Outside gRPC on the Inside
  30. 30. qaware.de QAware GmbH Aschauer Straße 32 81549 München Tel. +49 89 232315-0 info@qaware.de twitter.com/qaware linkedin.com/company/qaware-gmbh xing.com/companies/qawaregmbh slideshare.net/qaware github.com/qaware Upcoming Events: https://www.qaware.de/category/events/

×