Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Enhance system transparency and truthfulness with request tracing

424 views

Published on

Request tracing improves to the accessibility and discoverability of systems thus enhancing transparency.
This talk also shows a proof of concept real time rendering of a sequence diagram of a request using the tracing information.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Enhance system transparency and truthfulness with request tracing

  1. 1. REQUEST TRACING PROMOTING TRANSPARENCY & TRUTHFULNESS
  2. 2. Hello! I AM SAM KEEN polyglot software developer You can find me at: github.com/samkeen linkedin.com/in/samkeen @samkeen strava.com/athletes/560497
  3. 3. REQUEST TRACING What is it?
  4. 4. ASSOCIATING A GLOBALLY UNIQUE IDENTIFIER TO ALL REQUESTS OF A GIVEN SERVICE
  5. 5. COMMON OCCURRENCE IN THE WILD POST https://graph.facebook.com/v2.2/… Response headers … x-fb-rev → 1821576 x-fb-trace-id → E1+VUthAHKh x-fb-debug → NXN2ONVBqC8lLJVzgjknii86r5AJbmLGS5zX227q+jmLI8 sm+IIuAo8jETQ3bmH82w9SmjfAQktXTXpZJ7rtuA==
  6. 6. GREAT, LET’S SEE THE CODE!
  7. 7. ACTUALLY, I’D LIKE TO APPROACH THE SUBJECT IN A DIFFERENT WAY
  8. 8. BEFORE, WHAT, THERE IS WHY Development teams have a limited bandwidth Before launching an endeavor, estimate the value it will add to the company and use that to prioritize the work, or possibly rule it out
  9. 9. MEASURE VALUE How will the work help you obtain your goals
  10. 10. OK, GOALS, WHERE DO WE GET THOSE?
  11. 11. COMPLEXITY Software is one of the most complex of human endeavors To organize, prioritize goals, We need a plan
  12. 12. STRATEGIC GOALS These are the highest level, inward facing drivers for a company
  13. 13. ACQUIRE ONLINE SHOE SALES MARKET LEADER
  14. 14. PRINCIPLES Rules in place to help you remain aligned with Strategic Goals
  15. 15. SYSTEMS ARE BUILT WITH CONSISTENT, DOCUMENTED INTEGRATION POINTS ^ supports goal
  16. 16. PRACTICES Often technology specific guidance to ensure Principles are being upheld
  17. 17. ALL SERVICES ARE DOCUMENTED WITH API BLUEPRINT ^ supports principle
  18. 18. GOALS <PRINCIPLES <PRACTICES Strategic Goals + Acquire online shoe sales market leader Principles + Systems are built with consistent and documented integration points Practices + All services are documented with API Blueprint
  19. 19. GOALS <PRINCIPLES <PRACTICES Strategic Goals + Acquire online shoe sale market leader + Enable rapid innovation Principles + Systems are built with consistent and documented integration points + Ensure systems are transparent and discoverable Practices + All services are documented with API Blueprint + All application logs leverage trace tokens
  20. 20. PRINCIPLE: SYSTEM TRANSPARENCY what do we mean by that
  21. 21. NOT A NEW CONCEPT The Art of Linux Programming Chapter 6 : Transparency
  22. 22. SOFTWARE SYSTEM TRANSPARENCY GOOGLED - A strength of OS software is that the src code is available to all so the NSA cannot hide back doors in it. - DNS is a transparent system
  23. 23. A TRANSPARENT SOFTWARE SYSTEM - has a low learning curve, low barrier to entry - values simplicity over complexity - has high discoverability - is highly accessible
  24. 24. TRANSPARENCY SUPPORTING PRACTICES - gathering performance metrics - All devs have access to all repos - Any given system should be capable of running on a dev's machine (README) - centralized logging - Verbose modes of operation (-v). Have verbosity off by default but don't make it inaccessible
  25. 25. WTF, I THOUGHT THIS WAS A TALK ON TRACING? WHEN DO WE GET TO THE TRACING?
  26. 26. PRACTICE TO SUPPORT TRANSPARENCY PRINCIPLE Request Tracing
  27. 27. LOG ALL THE THINGS disk is cheap cpu is cheap logs can be valuable for more than stack traces
  28. 28. THE STEPS - per request create some sort of UUID token - override your logger to add said token to all log statements - add said token to all responses (in case of HTTP, use header)
  29. 29. ALTERNATIVES - Separate handler to its own log file - a TRACE level in the logger, just below or above DEBUG
  30. 30. DEMO
  31. 31. CLI aws --profile test logs filter-log- events --log-group-name export-demo --filter-pattern 559819cf1381f468e
  32. 32. SCALING The are many know strategies for scaling log aggregation Also many for log analysis For Instance: Real-time Processing of Log Data with Subscriptions [http://goo.gl/Oo1m73]
  33. 33. PRINCIPLE: TRUTHFULNESS How truthful is our documentation How do you create / maintain truthful documentation
  34. 34. TRUTHFUL DOCUMENTATION THE REALITY Static docs are at best: Truth’ish As soon as they are pened, they they start a steady migration towards lies and half- truths Entropy always wins
  35. 35. OFTEN THE CASE DOCS → MISINFORMATION planning stage, we document the system we plan to build. valuable exercise, spawns collaboration and feedback Then we build the system. Then the system ALWAYS changes. No one goes back and updates the docs
  36. 36. REDUCE PURE STATIC DOCS - Generated docs can help (javadoc, etc) - Real time metrics : better to look at what the app is actually doing, than rely on a perf test from 3 months ago Continual Non-functional testing anyone? - All docs should be wiki style, including diagrams!
  37. 37. DOCUMENTATION BEST PRACTICE Generate as much documentation as possible from the actual operation of the System and/or the source code of the System
  38. 38. PRACTICE TO SUPPORT TRUTHFUL DOCUMENTATION PRINCIPLE Request Rendering
  39. 39. DEMO
  40. 40. THANKS! Any questions? You can find me at: @samkeen / github.com/samkeen Slides:
  41. 41. CREDITS Special thanks to all the people who made and released these awesome resources for free: ○ Presentation template by SlidesCarnival ○ Photographs by Unsplash

×