ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
Modern webservices using gRPC and Protocol Buffers in Golang
1. Creating modern webservices
in
Golang
// Omid Hojabri
// E-mail: o.hojabri@gmail.com
// Skype: hojabri
// LinkedIn: linkedin.com/in/hojabri
using gRPC + Protocol
Buffers
2. REST : Popular API
REST (HTTP-JSON)
JSON
Format in URL:
GET /api/v1/user/231/roles/453
POST /api/v1/user/231/roles
PUT …
DELETE ...
3. Concerns about APIs
Data size
Latency
Scalability
Load Balancing
Communication between languages
Authentication
Monitoring
Logging
4. What is an API
Is a contract
Client sends REQUEST
Server sends RESPONSE
5. gRPC
Is a free and open-source framework developed by
Allows to simply define REQUEST and RESPONSE for RPC (Remote Procedure Calls)
Modern, Fast, Efficient (based on HTTP/2)
Low latency
Supports streaming
Many languages support it
Authentication, Monitoring, Load balancing and Logging are easy to define with gRPC
Many companies like Google, Netflix, Square, CoreOS, Spotify, Docker, Cisco,… using it fully.
6. Official supported languages
C/C++
Java
Android Java
Go
C#
Kotlin
Node.js
Objective-C
PHP (Beta)
Python
Ruby
Dart
7. RPC
Remote Procedure Call
Client calls functions directly on the server
gRPC implements RPC very cleanly and it solves a
lot of problems
8. gRPC overview
gRPC uses protocol buffers (protobuf) to communicate between clients and servers
9. How it runs?
gRPC needs to define messages and services using Protocol
Buffers
gRPC codes will be generated automatically with “protoc”
command!
Data is binary
You need just implementing the methods defined based on your
message type
Defining Protocol Buffers in .proto files.
10. .proto files
We define
messages and
service
definitions in
.proto files.
(API contract)
11. Types of API in gRPC
1 Unary
2 Server Streaming
3 Client Streaming
4 Bi-directional Streaming
15. Bi-directional Streaming API
Client Server
Client Message 1
Client Message 2
Client Message 3
Server Message 3
Server Message 4
Server Message n
Client Message 4
Client Message n
Server Message 2
Server Message 1
16. gRPC vs REST
gRPC REST
Data is binary, smaller and communication is faster (25
more performance based on benchmarks)
Text based, bigger and slower
HTTP/2 (Low latency) HTTP/1.1 (Higher latency)
Unary, Server streaming, Client streaming, bi-
directional streaming
Request – Response only
Any API commands CRUD (Create – Retrieve – Update – Delete) POST –
GET – PUT - DELETE
Code generation by protocol buffers in any language Manual code or by OpenAPI/Swagger
RPC based HTTP words based
17. Installation
Prerequisites:
Go to be installed (any one of the three latest major
release)
Protocol Buffer compiler (protoc v3)
Go plugin for protocol compiler
18. Protocol Buffer Compiler (v3)
The protocol buffer compiler (protoc) is used to
compile (code generation) .proto files, which contain
service and message definitions.
Linux:
MacOS:
19. Protocol Buffer Compiler (v3)
Windows:
Download and install the latest release of binary version
from:
https://github.com/protocolbuffers/protobuf/releases/tag/v3.13.0
(at the moment v3.13.0 is released)
20. Go plugin for the protocol
buffers
1 Install the protocol compiler plugin for Go (protoc-gen-go)
using the following command:
2 Update your PATH so that the protoc compiler can find the
plugin: