Observability has become a buzzword, and the API world is slowly catching up. But be careful when looking for content about API observability- there are many outdated best practices and irrelevant pieces of content out there. Based on the blog: https://tyk.io/blog/bad-api-observability/

1. 1
KCD AUSTRIA September 26th
-27th
2023
Bad API
Observability
Sonja Chevre
Tyk Technologies

2. This talk was inspired by a conversation with
Stephen Townshend from SlightReliability

3. Why is API
Observability
a thing?

4. APIs have become building
blocks of the digital world
01

5. APIs are products
that generate revenue
02

6. APIs demand
cross-functional teamwork
03

7. A simple example
API gateway
with OTel built-in
API users
Authentication
Authorization
API versioning
Rate-limiting
Request/response
transformation
Built-in telemetry
Services

8. Observability stack
spans
API gateway
with OTel built-in
Services
Collector
Span
metrics
connector
spans

9. observability
anti-patterns
to avoid when
working with
APIs
7

10. Anti-pattern #1
relying on API monitoring only

11. API Monitors
API Monitoring = Black box testing
API gateway
with OTel built-in
API Monitors
Services
API Monitors

12. API Monitoring
For uptime, availability, SLAs, …
All good!

13. The reality

14. Looks better, right?

15. Anti-pattern #2
you forgot your users

16. Doesn’t look
good anymore

17. 400
401
404
405
Bad Request
Unauthorized
Not Found
Method Not Allowed
Monitor client error responses
For better developer experience,

18. Who are your users?
What problems do
they face?
Answering those
questions allow you
to shift from reactive
to proactive.
Metrics per API users

19. Anti-pattern #3
using API access logs for
troubleshooting

22. No more guesswork

23. Anti-pattern #4
Starting the trace at the
API gateway is overrated

24. A lot of room for errors!
(rate-limiting, authentication, …)

25. Anti-pattern #5
different teams, different tools,
different data

26. Centralized API platform with multi-teams
API gateway
with OTel built-in
Users
Service B
(team 2)
Service A
(team 1)
spans
spans
spans

27. Observability pipelines to the rescue
spans
Collector
Telemetry
pipeline
API platform team
All the spans
Dev team 1
Spans for their APIs
Dev team 2
Spans for their APIs

28. OTel Collector
config to send
the same spans
to multiple
observability
back-ends
Telemetry pipelines

29. Processors
https://github.com/open-telemetry/opentelemetry
-collector-contrib/tree/main/processor

30. Anti-pattern #6
one size fits all

31. REST is still very popular,
but not the only option

32. Remember this?

33. GraphQL errors are handled differently
The request returns HTTP
status code 200 even though
the response contains an error
Here is the error!

34. GraphQL errors are
not reported on
the span

35. Report the error to
the active span
with manual
instrumentation

36. The error is now
recorded in the span
Including error details

37. Anti-pattern #7
observability is for production only

38. Increase
collaboration
between teams
OBSERVABILITY FOR DEVS
Add JSON trace
in Jira ticket

41. Your availability is irrelevant if your users are upset
Forget about logs for troubleshooting API requests
Leverage OTel to break silos between teams
Understand your tech stack (REST, GraphQL, gRPC, …)
Users first
SUM IT UP

42. 42
Thank you
https://www.linkedin.com/in/sonjachevre/
https://twitter.com/SonjaChevre