Simplifying the OpenAPI
Development Experience
Cody A. Ray, Staff SWE
Tech Lead Manager, API Foundations
API World - Oct 27, 2021
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Assumptions
1. You’re amped up about API contract-first design
2. You’ve decided to standardize on OpenAPI specifications
3. ???
4. Profit
Today’s talk will focus on the tactical steps for adopting OpenAPI
contract-first design in practice.
2
API Design Guide
Rules to ensure all
your teams make
high-quality and
consistent APIs
Minispec
A new DSL and
generator for
OpenAPI specs
OpenAPI Challenges
Deep dive into some
challenges to using
OpenAPI “right” that
Minispec handles
API Linters
Minispec gets you
70%+ of the way.
This is helping us
close another 20%+
API Design Guidelines
5
API Design Guides
Hyper-growth companies be like
● Q1 - 1 API
● Q2 - 5 APIs
● Q3 - 7 APIs
When you know that’s coming, you do your best
to “get ahead of it” by adopting a company-wide
API Design Guide in advance.
Tons of company-specific design guides but no
real standards for REST API design.
6
API Design Standards?
Ok, maybe {JSON:API} (jsonapi.org)
We tried hard but failed to adopt it.
1. Designed for requirements we don’t have
and never expect to adopt.
a. Ex: Compound Documents.
“Why not just GraphQL?”
2. Inflexibility for requirements we do have
a. Not self-descriptive (api_version and kind)
b. No batch/bulk support
3. Developer pushback for unusual relationship
handling (e.g., related links can’t change)
4. PM pushback on verbosity (tech products)
5. Low adoption and development seems slow
(1.1 stalled since 2015)
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
A New Standard!
7
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
A New Problem!
8
Or maybe 1000 new problems.
Even the existing “standards” and inspiration
left a ton of gaps we needed to fill ourselves.
And try to convince 100s of busy developers
who are already working on hard problems
to also internalize these API nuances is…
impossible.
Minispec
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Minispec
10
DSL for REST APIs
Let developers focus on the
bits they (should) care
about.
● Resources, attributes
and relationships
● Filters, pagination, and
sorting (on list requests)
● Versioning, deprecations
NOT urls, methods, query
params, request and
response bodies, etc.
Generates OpenAPI
On average, the minispec
DSL for an API is 1/5th the
size of the final OpenAPI.
More importantly, it
handles all the details and
matches the design guide.
Aggregator and Filters
Multiple APIs, some
supporting different
audiences (external, UI/CLI)
Different use cases require
different OpenAPI specs. 😩
● Lack of support for
“immutable”, “create
only” etc in OpenAPI
● AllOf bugs in the
OpenAPI-generator
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
But… GraphQL? GRPC?
11
This is more a business question than a technical one
● How broad is the market you’re targeting?
● Who are the customers?
● How technically savvy are they?
● What technologies are they comfortable with?
Confluent serves everyone from Fortune 500 companies to one-person
shops. We want the broadest possible adoption with the minimal
possible friction. Everyone knows HTTP. #businessdecision
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
The Many Specs of OpenAPI
12
13
Minispec DSL
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Minispec DSL
14
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Minispec DSL
15
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Wait, what is this doing?
16
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Too much code. Pictures please!
17
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. 18
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Generated OpenAPI
19
OpenAPI Challenges
21
OpenAPI Challenges
● Required Fields
● Immutable Fields
● Read-Once Fields (e.g., creating a secret)
● Context-Aware Examples
● OneOf/AllOf/AnyOf support
● Shared Schemas (e.g., BadRequest, Error)
● ...
22
Required | Immutable
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Required Fields - Bugs
23
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Read-Once Fields - Not Supported
24
Return the API Key secret only once, when its first created.
Ideally openapi-generator support to automatically remove this value from marshalling
(e.g., non-create API operations, customer-facing audit log and events, developer logs, etc).
25
Contextual Examples
Have to override examples on common schemas
for every operation.
Ex: standardizing request or resource metadata:
Another “allOf”
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. 26
AllOf / AnyOf / OneOf - Bugs
OpenAPI Linter
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
80+ custom rules in Spectral (from Spotlight)
28
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Automating API
PR Reviews
29
Eventually we hope to remove
centralized API team reviews.
Make API design entirely self-service
But I’m not sure if that’s possible.
Linters are good at low-level details.
Not at big picture design:
“Should this be a resource?”
“How should we model this?”
API Design Guide
Rules to ensure all
your teams make
high-quality and
consistent APIs
Minispec
A new DSL and
generator for
OpenAPI specs
OpenAPI Challenges
Deep dive into some
challenges to using
OpenAPI “right” that
Minispec handles
API Linters
Minispec gets you
70%+ of the way.
This is helping us
close another 20%+
Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Vision: API Publishing Platform
31
cnfl.io/careers cnfl.io/slack
cnfl.io/blog
Thank you!
@codyaray
cody@confluent.io
PS - We’re Hiring! :D

Simplifying the OpenAPI Development Experience

  • 1.
    Simplifying the OpenAPI DevelopmentExperience Cody A. Ray, Staff SWE Tech Lead Manager, API Foundations API World - Oct 27, 2021
  • 2.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Assumptions 1. You’re amped up about API contract-first design 2. You’ve decided to standardize on OpenAPI specifications 3. ??? 4. Profit Today’s talk will focus on the tactical steps for adopting OpenAPI contract-first design in practice. 2
  • 3.
    API Design Guide Rulesto ensure all your teams make high-quality and consistent APIs Minispec A new DSL and generator for OpenAPI specs OpenAPI Challenges Deep dive into some challenges to using OpenAPI “right” that Minispec handles API Linters Minispec gets you 70%+ of the way. This is helping us close another 20%+
  • 4.
  • 5.
    5 API Design Guides Hyper-growthcompanies be like ● Q1 - 1 API ● Q2 - 5 APIs ● Q3 - 7 APIs When you know that’s coming, you do your best to “get ahead of it” by adopting a company-wide API Design Guide in advance. Tons of company-specific design guides but no real standards for REST API design.
  • 6.
    6 API Design Standards? Ok,maybe {JSON:API} (jsonapi.org) We tried hard but failed to adopt it. 1. Designed for requirements we don’t have and never expect to adopt. a. Ex: Compound Documents. “Why not just GraphQL?” 2. Inflexibility for requirements we do have a. Not self-descriptive (api_version and kind) b. No batch/bulk support 3. Developer pushback for unusual relationship handling (e.g., related links can’t change) 4. PM pushback on verbosity (tech products) 5. Low adoption and development seems slow (1.1 stalled since 2015)
  • 7.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. A New Standard! 7
  • 8.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. A New Problem! 8 Or maybe 1000 new problems. Even the existing “standards” and inspiration left a ton of gaps we needed to fill ourselves. And try to convince 100s of busy developers who are already working on hard problems to also internalize these API nuances is… impossible.
  • 9.
  • 10.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Minispec 10 DSL for REST APIs Let developers focus on the bits they (should) care about. ● Resources, attributes and relationships ● Filters, pagination, and sorting (on list requests) ● Versioning, deprecations NOT urls, methods, query params, request and response bodies, etc. Generates OpenAPI On average, the minispec DSL for an API is 1/5th the size of the final OpenAPI. More importantly, it handles all the details and matches the design guide. Aggregator and Filters Multiple APIs, some supporting different audiences (external, UI/CLI) Different use cases require different OpenAPI specs. 😩 ● Lack of support for “immutable”, “create only” etc in OpenAPI ● AllOf bugs in the OpenAPI-generator
  • 11.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. But… GraphQL? GRPC? 11 This is more a business question than a technical one ● How broad is the market you’re targeting? ● Who are the customers? ● How technically savvy are they? ● What technologies are they comfortable with? Confluent serves everyone from Fortune 500 companies to one-person shops. We want the broadest possible adoption with the minimal possible friction. Everyone knows HTTP. #businessdecision
  • 12.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. The Many Specs of OpenAPI 12
  • 13.
  • 14.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Minispec DSL 14
  • 15.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Minispec DSL 15
  • 16.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Wait, what is this doing? 16
  • 17.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Too much code. Pictures please! 17
  • 18.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. 18
  • 19.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Generated OpenAPI 19
  • 20.
  • 21.
    21 OpenAPI Challenges ● RequiredFields ● Immutable Fields ● Read-Once Fields (e.g., creating a secret) ● Context-Aware Examples ● OneOf/AllOf/AnyOf support ● Shared Schemas (e.g., BadRequest, Error) ● ...
  • 22.
  • 23.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Required Fields - Bugs 23
  • 24.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Read-Once Fields - Not Supported 24 Return the API Key secret only once, when its first created. Ideally openapi-generator support to automatically remove this value from marshalling (e.g., non-create API operations, customer-facing audit log and events, developer logs, etc).
  • 25.
    25 Contextual Examples Have tooverride examples on common schemas for every operation. Ex: standardizing request or resource metadata: Another “allOf”
  • 26.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. 26 AllOf / AnyOf / OneOf - Bugs
  • 27.
  • 28.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. 80+ custom rules in Spectral (from Spotlight) 28
  • 29.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Automating API PR Reviews 29 Eventually we hope to remove centralized API team reviews. Make API design entirely self-service But I’m not sure if that’s possible. Linters are good at low-level details. Not at big picture design: “Should this be a resource?” “How should we model this?”
  • 30.
    API Design Guide Rulesto ensure all your teams make high-quality and consistent APIs Minispec A new DSL and generator for OpenAPI specs OpenAPI Challenges Deep dive into some challenges to using OpenAPI “right” that Minispec handles API Linters Minispec gets you 70%+ of the way. This is helping us close another 20%+
  • 31.
    Copyright 2021, Confluent,Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. Vision: API Publishing Platform 31
  • 32.