Request tracing improves to the accessibility and discoverability of systems thus enhancing transparency. This talk also shows a proof of concept real time rendering of a sequence diagram of a request using the tracing information.

1. REQUEST
TRACING
PROMOTING
TRANSPARENCY &
TRUTHFULNESS

2. Hello!
I AM SAM KEEN
polyglot software developer
You can find me at:
github.com/samkeen
linkedin.com/in/samkeen
@samkeen
strava.com/athletes/560497

3. REQUEST TRACING
What is it?

4. ASSOCIATING A GLOBALLY
UNIQUE IDENTIFIER TO ALL
REQUESTS OF A GIVEN SERVICE

5. COMMON OCCURRENCE IN THE
WILD
POST https://graph.facebook.com/v2.2/…
Response headers
…
x-fb-rev → 1821576
x-fb-trace-id → E1+VUthAHKh
x-fb-debug →
NXN2ONVBqC8lLJVzgjknii86r5AJbmLGS5zX227q+jmLI8
sm+IIuAo8jETQ3bmH82w9SmjfAQktXTXpZJ7rtuA==

6. GREAT, LET’S SEE THE CODE!

7. ACTUALLY,
I’D LIKE TO APPROACH THE
SUBJECT IN A DIFFERENT WAY

8. BEFORE, WHAT, THERE IS WHY
Development teams have a limited
bandwidth
Before launching an endeavor, estimate the
value it will add to the company and use
that to prioritize the work, or possibly rule it
out

9. MEASURE VALUE
How will the work help you obtain your
goals

10. OK, GOALS, WHERE DO WE GET
THOSE?

11. COMPLEXITY
Software is one of the most complex of
human endeavors
To organize, prioritize goals, We need a plan

12. STRATEGIC GOALS
These are the highest level, inward facing
drivers for a company

13. ACQUIRE ONLINE SHOE SALES
MARKET LEADER

14. PRINCIPLES
Rules in place to help you remain aligned
with Strategic Goals

15. SYSTEMS ARE BUILT WITH
CONSISTENT, DOCUMENTED
INTEGRATION POINTS
^ supports goal

16. PRACTICES
Often technology specific guidance to ensure
Principles are being upheld

17. ALL SERVICES ARE DOCUMENTED
WITH API BLUEPRINT
^ supports principle

18. GOALS <PRINCIPLES <PRACTICES
Strategic Goals
+ Acquire online
shoe sales market
leader
Principles
+ Systems are built
with consistent
and documented
integration points
Practices
+ All services are
documented with
API Blueprint

19. GOALS <PRINCIPLES <PRACTICES
Strategic Goals
+ Acquire online
shoe sale market
leader
+ Enable rapid
innovation
Principles
+ Systems are built
with consistent
and documented
integration points
+ Ensure systems
are transparent
and discoverable
Practices
+ All services are
documented with
API Blueprint
+ All application
logs leverage
trace tokens

20. PRINCIPLE:
SYSTEM TRANSPARENCY
what do we mean by that

21. NOT A NEW CONCEPT
The Art of Linux Programming
Chapter 6 : Transparency

22. SOFTWARE SYSTEM
TRANSPARENCY
GOOGLED
- A strength of OS software is that the src
code is available to all so the NSA cannot
hide back doors in it.
- DNS is a transparent system

23. A TRANSPARENT SOFTWARE
SYSTEM
- has a low learning curve, low barrier to
entry
- values simplicity over complexity
- has high discoverability
- is highly accessible

24. TRANSPARENCY SUPPORTING
PRACTICES
- gathering performance metrics
- All devs have access to all repos
- Any given system should be capable of
running on a dev's machine (README)
- centralized logging
- Verbose modes of operation (-v). Have
verbosity off by default but don't make it
inaccessible

25. WTF, I THOUGHT THIS WAS A TALK
ON TRACING?
WHEN DO WE GET TO THE
TRACING?

26. PRACTICE TO SUPPORT
TRANSPARENCY PRINCIPLE
Request Tracing

27. LOG ALL THE THINGS
disk is cheap
cpu is cheap
logs can be valuable for more than stack
traces

29. THE STEPS
- per request create some sort of UUID
token
- override your logger to add said token to
all log statements
- add said token to all responses (in case
of HTTP, use header)

30. ALTERNATIVES
- Separate handler to its own log file
- a TRACE level in the logger, just below or
above DEBUG

31. DEMO

32. CLI
aws --profile test logs filter-log-
events
--log-group-name export-demo
--filter-pattern 559819cf1381f468e

34. SCALING
The are many know strategies for scaling
log aggregation
Also many for log analysis
For Instance:
Real-time Processing of Log Data with
Subscriptions [http://goo.gl/Oo1m73]

35. PRINCIPLE: TRUTHFULNESS
How truthful is our documentation
How do you create / maintain truthful
documentation

36. TRUTHFUL DOCUMENTATION
THE REALITY
Static docs are at best: Truth’ish
As soon as they are pened, they they start a
steady migration towards lies and half-
truths
Entropy always wins

37. OFTEN THE CASE
DOCS → MISINFORMATION
planning stage, we document the system
we plan to build.
valuable exercise, spawns collaboration and
feedback
Then we build the system. Then the system
ALWAYS changes.
No one goes back and updates the docs

38. REDUCE PURE STATIC DOCS
- Generated docs can help (javadoc, etc)
- Real time metrics : better to look at what
the app is actually doing, than rely on a
perf test from 3 months ago
Continual Non-functional testing anyone?
- All docs should be wiki style, including
diagrams!

39. DOCUMENTATION BEST PRACTICE
Generate as much documentation as
possible from the actual operation of the
System and/or the source code of the
System

40. PRACTICE TO SUPPORT TRUTHFUL
DOCUMENTATION PRINCIPLE
Request Rendering

42. DEMO

43. THANKS!
Any questions?
You can find me at:
@samkeen / github.com/samkeen
Slides:

44. CREDITS
Special thanks to all the people who made and released these awesome
resources for free:
○ Presentation template by SlidesCarnival
○ Photographs by Unsplash