This document discusses building APIs with Swift, OpenAPI, and gRPC. It introduces protocol buffers for defining data structures, and gRPC for building APIs. It recommends using the gnostic tool to convert OpenAPI descriptions to protocol buffers for use with gRPC plugins. This allows building high-quality code generators in different languages by separating the generator from the API description parsing. The document provides examples of building gRPC APIs and clients in Swift.

1. Build Great Networked APIs
with Swift, OpenAPI, and gRPC
Tim Burks, Google
Swift Cloud Workshop No. 2
September 30, 2017

2. What do we do?

3. WE WRITE CODE

5. What do we want?

6. QUALTIY!

7. Protocol Buffers
a language-neutral, platform-neutral, extensible mechanism for serializing structured data.

8. Interface Builder for Data
message Person {
string name = 1;
int32 id = 2;
string email = 3;
message PhoneNumber {
string number = 1;
}
repeated PhoneNumber phone = 4;
}
Interface Builder: Developers specify
their interfaces using a special tool,
tooling compiles and integrates that into
their apps.
Protocol Buffers: Developers specify
their data structures using a special
language, tooling compiles and
integrates that into their apps.

9. Google API Discovery Format

10. Discovery Format Code Generators

11. github.com/google/google-api-go-client

12. github.com/google/google-api-python-client

13. Idiosyncracies and Lessons

14. Toolkit and the next generation of Google APIs

15. So far, so good.
REST (Discovery Format)
● 232 API descriptions at https://www.googleapis.com/discovery/v1/apis
● 10 generators listed at https://developers.google.com/api-client-library/
gRPC (Toolkit)
● 30+ gRPC-based APIs
● 7 target languages

16. OpenAPI 3.0
Industry standard format for describing for REST APIs
Originally designed for documentation, now with many other applications: API
Authoring, Validation, Documentation, Analysis, Search, Testing, Mocking,
Management, Code Generation

17. Consensus

18. OpenAPI-based code generators
● swagger-codegen (Open source, Smartbear + community)
○ 70+ targets
○ First commit: July 6, 2011
○ Used by Lyft and Square to generate SDKs
● AutoRest (Open source, Microsoft)
● oas-nodegen (Open source, Capital One)
● APIMatic (Proprietary, APIMatic)
more?

20. Problems we’ve seen so far in open source code generators
● Missing or weak build system integration.
● Invalid service addresses (error in OpenAPI description).
● No auth.
● No documentation (not even a README!).
● No samples.
● No test harness.
● No assurance that the called service even works.
● Ugly generated code.
○ Machine-generated operation names.
○ Machine-generated struct names.
● Unwanted dependencies.

21. swagger-codegen often isn’t used “out of the box”:
“Generating client libraries involves customizing the provided language-specific
templates…
The amount of modification each template needs varies by language and we’re
looking forward to working with the Swagger Codegen community to share our
refinements.” Val Polouchkine, Lyft
“...Swagger Codegen is a pretty active project. If you don’t check in your
templates, things are gonna break unexpectedly because Swagger Codegen just
uses the latest and greatest templates that are out there. So if you don’t sort of
manually make sure that those things work, you’re gonna have an issue there.”
Tristan Sokol, Square

22. Code generation pipeline
API Description
Generatable
API Description
Language-
specific model
Generated
API support
code
render idiomatic API
support code according
to user preferences
(re. build systems,
dependencies)
verify model,
define structures and
entry points,
name everything
filter language-
specific reserved words,
(optionally) define file
structure for generated code

23. typical monolithic
code generation pipeline
API Description
(OpenAPI)
Generatable
API Description
(internal data
structures)
Language-
specific model
(internal data
structures)
Generated
API support
code
All-in-one repo and package

24. Problems with monolithic code generators
● Long build times: changing one target requires rebuilding everything.
● Long test times: new builds must be tested for every target language.
● For stability, teams may prefer to archive their own generator builds.
● Forks will abound.
● Quality is uneven.
● Versioning is hard.
● Complexity and potentially unfamiliar build systems deter contributors.

25. protoc
code generation pipeline
API Description
(.proto)
Generatable
API Description
(binary
FileDescriptors)
Language-
specific model
(internal data
structures)
Generated
API support
code
language target-
specific plugins
protoc

26. protoc plugin definition

27. Why does protoc have a plug-in architecture?
● Fast build times: changing one target only requires rebuilding its plugin.
● Fast test times: new builds need only be tested for the affected targets.
● For stability, teams can archive their own protoc and plugin builds.
● New plugins can abound.
● Separately-maintained plugins can offer different maturity levels.
● Separately-maintained plugins can be appropriately versioned.
● Separately-maintained plugins can be in languages that contributors
prefer.

28. What’s the catch?
Plugins require a well-defined interchange format.
Fortunately, we have two great tools for that.

29. gnostic
gnostic processed and
verified protobuf
representation of
OpenAPI description
protoc +
pluginsOpenAPI
.proto
reusable data structures
and reader for protobuf
OpenAPI descriptions
gnostic apps
and plugins
OpenAPI
description
gnostic-
generator
OpenAPI
.proto and
compiler
code
OpenAPI
JSON
schema

30. Kubernetes OpenAPI: .json vs .pb
Format Size Deserialization time Download time
(at 80 Mbps)
Json 1653 KB >500 ms 165.3 ms
Proto binary 914 KB 9.3 ms 91.4 ms
Proto binary compressed 96 KB 13.5 ms 1.3 ms
Source: mehdy@google.com

31. gnostic
code generation pipeline
API Description
(OpenAPI v3)
Generatable
API Description
(Normalized/
Annotated
OpenAPI v3?)
Language-
specific model
(Normalized/
Annotated
OpenAPI v3?)
Generated
API support
code
target-specific
plugins
gnostic +
linter

32. let anyone write code generation plugins...
● in their own repositories
● with their own versioning
● in whatever implementation language they choose
API code generation is a community problem that needs community-based
solutions...

33. Code generators should
belong to their communities.

34. gRPC

35. @grpcio
RPC: Use Cases
Direct RPCs:
Microservices
RPCs to
access APIs
Google APIs
OSS APIs
Mobile
Web
Desktop
RPCs
Datacenters Cloud
Service 1
Service 2
Service 3
Service 4
Containers

36. @grpcio
What is gRPC?
● HTTP/2 based RPC framework developed by Google
● Open, Multiplatform, Secure, Performant
Multiplatform
● Idiomatic APIs in popular languages (C++, Go, Java, Python, Node.js, C#, Ruby, PHP)
● Supports mobile devices (Android Java, iOS Obj-C, Swift)
● Linux, Windows, Mac OS X
● (web browser support in development)
OpenSource
● developed fully in open on GitHub: https://github.com/grpc/

37. @grpcio
● Builds on Apple’s swift-protobuf and grpc-core.
● Includes:
○ gRPC framework (C and Swift components)
○ generated code surface
○ protoc plugin for code generation
● Full-service gRPC:
○ All four gRPC API styles are supported.
○ gRPC framework supports both clients and servers.
○ Plugin generates client and server code in separate files.
○ Testing on MacOS and Ubuntu.
● Audiences: client and server developers, inside/outside Google
gRPC for Swift (https://github.com/grpc/grpc-swift)

38. @grpcio
package echo;
service Echo {
// Immediately returns an echo of a request.
rpc Get(EchoRequest) returns (EchoResponse) {}
// Splits a request into words and returns each word in a stream of messages.
rpc Expand(EchoRequest) returns (stream EchoResponse) {}
// Collects a stream of messages and returns them concatenated when the caller closes.
rpc Collect(stream EchoRequest) returns (EchoResponse) {}
// Streams back messages as they are received in an input stream.
rpc Update(stream EchoRequest) returns (stream EchoResponse) {}
}
message EchoRequest {
// The text of a message to be echoed.
string text = 1;
}
message EchoResponse {
// The text of an echo response.
string text = 1;
}
gRPC Swift sample service

39. @grpcio
/// To build a server, implement a class that conforms to this protocol.
public protocol Echo_EchoProvider {
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws -> Echo_EchoResponse
func expand(request : Echo_EchoRequest, session : Echo_EchoExpandSession) throws
func collect(session : Echo_EchoCollectSession) throws
func update(session : Echo_EchoUpdateSession) throws
}
gRPC Swift server protocol

40. @grpcio
// get returns requests as they were received.
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws -> Echo_EchoResponse {
return Echo_EchoResponse(text:"Swift echo get: " + request.text)
}
...
// update streams back messages as they are received in an input stream.
func update(session : Echo_EchoUpdateSession) throws -> Void {
while true {
do {
let request = try session.Receive()
try session.Send(Echo_EchoResponse(text:"Swift echo update: (request.text)"))
} catch Echo_EchoServerError.endOfStream {
break
}
}
try session.Close()
}
}
gRPC Swift server sample

41. @grpcio
gRPC Swift unary client sample
// Unary
if client == "get" {
var requestMessage = Echo_EchoRequest(text:message)
let responseMessage = try service.get(requestMessage) // blocking
print("get received: " + responseMessage.text)
}

42. @grpcio
gRPC Swift bidirectional streaming client sample (1/2)
// Bidirectional streaming
if client == "update" {
let sem = DispatchSemaphore(value: 0)
let updateCall = try service.update() // blocking
DispatchQueue.global().async {
while true {
do {
let responseMessage = try updateCall.Receive() // blocking
print("Received: (responseMessage.text)")
} catch Echo_EchoClientError.endOfStream {
sem.signal()
break
}
}
}
...

43. @grpcio
gRPC Swift bidirectional streaming client sample (2/2)
...
let parts = message.components(separatedBy:" ")
for part in parts {
let requestMessage = Echo_EchoRequest(text:part)
try updateCall.Send(requestMessage)
sleep(1)
}
try updateCall.CloseSend()
// Wait for the call to complete.
sem.wait()
}

45. gRPC-Swift TODO
● Build system integration
○ Package Manager
○ Cocoapods?
○ Carthage?
● gRPC interoperability tests
● Samples that wrap Google APIs
○ Google Cloud Speech API
○ Google Datastore API
github.com/grpc/grpc-swift/issues

46. More gRPC Information
Website: http://grpc.io
Sources: https://github.com/grpc/grpc
Mailing list: https://groups.google.com/d/forum/grpc-io
Ecosystem: https://github.com/grpc-ecosystem

47. github.com/googleapis/gnostic
plugins for OpenAPI-based code generation
github.com/grpc/grpc-swift
Fast streaming APIs in Swift
github.com/google/auth-library-swift
OAuth support for Google Cloud
http://twitter.com/timburks
timburks@google.com