Have you ever left out a comma in your payload calling an API on the command line? Have you cURL’d a REST API only to realize you forgot that crucial query parameter? If only the command line could take advantage of some structured description of the API model… APIs can be challenging to understand and consume from the command line, a utility most developers use daily. In this session we will look at an approach to generating intuitive command line utilities for gRPC APIs.

1. Intuitive CLIs for gRPC APIs
Noah Dietz
(@no_d_here)

2. curl https://my.api.net/post
-H "Content-Type: application/json"
-d '{"requird": "field"}'

3. curl https://my.api.net/post
-H "Content-Type: application/json"
-d '{"required": "field"}'

4. What it feels like today
Web Interface Command-line API

5. Perfect World
Web Interface Command-line API

6. Likely Reality
Web Interface Command-line APIAPI

7. Q: Why do we care about command line
consumption?

8. A: It’s a gateway for new developers and
consumers

9. Q: How can we improve the command line
API consumption experience?

10. A: Use structured definitions to intuitively
represent the interface

11. Protocol Buffers
Code-gen
Source

12. Protocol Buffers
IDL (Interface definition language)
Describe once and generate interfaces for any
language.
Data Model
Structure of the request and response.
Wire Format
Binary format for network transmission.
service RouteGuide {
rpc GetFeature(Point) returns (Feature);
rpc RouteChat(stream RouteNote) returns (stream RouteNote);
}
message Point {
int32 latitude = 1;
int32 longitude = 2;
}
message Feature {
string name = 1;
Point location = 2;
}
message RouteNote {
Point location = 1;
string message = 2;
}

13. C/C++
Java is a registered trademark of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

14. Cutting edge transport
Bespoke wire format
Consumed via generated code
How from CLI?!

15. State of the Art
● Server reflection-based CLIs
○ Similar to HTTP cURL
○ Favors flexibility over API
consumption experience
● github.com/grpc/grpc/.../grpc_cli
● github.com/fullstorydev/grpcurl
#grpcurl example
# listing available Services
> grpcurl my.api.net list
MyService
# listing Service RPCs
> grpccurl my.api.net list MyService
MethodA
MethodB
# describing specific Service RPC
> grpcurl my.api.net describe MyService MethodA
…
# describing specific Message
> grpcurl my.api.net describe MethodAInput
…
# invoking a Service RPC
> grpcurl -d '{"name": "Jane Doe"}'
my.api.net
MyService.MethodA
…

16. Handwritten CLIs?

18. github.com/googleapis/kiosk

19. “Why wasn’t this CLI generated?”
- @timburks

21. @no_d_here
Generated
Command Line
Interfaces

22. Generated Command Line Interfaces (gcli)
● Go lang
● Cobra CLI framework
● Uses Google generated API clients (GAPIC)
● github.com/googleapis/gapic-generator-go

23. Goals
● Generate a CLI tailored to a given gRPC
API
● Generate a more human-oriented
interface
● Unified experience across consumption
mediums
Non-Goals
● Provide a single, generic utility that can
invoke any gRPC API
● Replace gcloud SDK as the canonical
Google Cloud CLI

24. Generate a CLI tailored to
a given gRPC API

25. API Description → Command Structure
kctl display list-kiosks
Root command Service name Method name

26. API Description → Command Structure
$ kctl display
Top level command for Service: Display
Usage:
kctl display [command]
Available Commands:
create-kiosk
create-sign
delete-kiosk
delete-sign
get-kiosk
get-sign
list-kiosks
list-signs
...

27. Payload construction: Primitives
kctl display get-sign --id=<int32>
Field name Field type

28. kctl ... --kiosk_ids=<int32> --kiosk_ids=<int32>
Repeated field name Repeated field type
Payload construction: Repeated Primitives

29. kctl ... --size.width=<int32> --size.height=<int32>
Nested
field’s
name
Nested
message
field
name
Nested
message
field
type
Payload construction: Nested Messages

30. “Correct by construction”
- @timburks

31. Generate a more
human-oriented interface

32. Making machine things human: LRO
kctl display cycle-kiosks --id=123 --follow
Wait for LRO
to finish
kctl display cycle-kiosks-poll --operation=<id>
Operation IDExtra polling
helper

33. Making machine things human: Streaming
● stdin OR
--from_file=<path>
○ JSON Protobuf format
● stdout OR
--out_file=<path>
# JSON Protobuf input format
{"field": "a"}
{"field": "b"}
{"field": "c"}
{"field": "d"}

34. Unified experience
across consumption
mediums

35. Educate the consumer: --help
$ kctl display create-sign --help
Create a sign. This enrolls the sign for sign display.
Usage:
kctl display create-sign [flags]
Flags:
--from_file string Absolute path to JSON file containing request payload
-h, --help help for create-sign
--image bytesHex Data of image currently displayed on sign
--name string Required. Name of the sign
--text string Text to display on sign
...

36. Educate the consumer: bash autocompletion
$ kctl completion --help
Enable bash completion like so:
Linux:
source <(kctl completion)
Mac:
brew install bash-completion
kctl completion > $(brew --prefix)/etc/bash_completion.d/kctl
Usage:
kctl completion [flags]
Flags:
-h, --help help for completion

37. kctl/
create-kiosk.go
delete-sign.go
get-sign.go
list-kiosks.go
create-sign.go
display_service.go
list-signs.go
delete-kiosk.go
get-kiosk.go
root.go
Educate the consumer: code & structure
Service command
root command
Method command
# Input message to generated usage
message CreateKioskRequest →
var CreateKioskInput *pb.CreateKioskRequest
# RPC to generated cmd to usage
rpc CreateKiosk → var createKioskCmd →
kctl display create-kiosk

38. @no_d_here
Demo

39. @no_d_here
When a CLI won’t
help

40. Large, nested payloads
--consistency_selector.new_transaction.mode.read_only.consistency_selector.read_time.seconds
--consistency_selector.new_transaction.mode.read_only.consistency_selector.read_time.nanos
firestore.Firestore.BatchGetDocuments
dataproc.WorkflowTemplateService.CreateWorkflowTemplate
● Had > 50 flags
● Deeply nested fields

41. Unstructured “structured” APIs
curl https://my.api.net/post
-H "Content-Type: application/json"
-d '{"sql": "SELECT * FROM foo;"}'

42. @no_d_here
All this to say...

43. Wrapping it up
● CLI API consumption is a gateway & important tool
● Structured API definitions should guide the CLI experience
● Educate the consumer at every opportunity
● Not every API is meant for the command line

44. Thank you
Noah Dietz
(@no_d_here)