14. possible approaches
• Unique identifier
• propagate throughout
• write instrumentation for various
transports
• Observe and correlate
15. possible approaches
• Unique identifier
• propagate throughout
• write instrumentation for various
transports
• Observe and correlate
• always on the outside - black box
16. possible approaches
• Unique identifier
• propagate throughout
• write instrumentation for various
transports
• Observe and correlate
• always on the outside - black box
• difficult to get threaded + evented
processes right
19. t = start
nginx
cache python
db internet
the java
20. t = start
nginx
cache python
db internet
the java
21. t = start
nginx
cache python
db internet
the java
22. t = start
nginx
cache python
db internet
the java
23. t = start
nginx
cache python
db internet
the java
24. t = start
nginx
cache python
db internet
the java
25. t = start
nginx
cache python
db internet
the java
26. t = start
nginx
cache python
db internet
the java
27. t = start
nginx
cache python
db internet
the java
28. t = start
nginx
cache python
db internet
the java
29. t = start
nginx
cache python
db internet
the java
30. t = start
nginx
cache python
db internet
the java
31. t = start
nginx
cache python
db internet
the java
32. t = start
nginx
cache python
db internet
the java
33. t = start
nginx
cache python
db internet
the java
t = end
34. piggyback rides
• More Doable
• HTTP: x-headers
• Thrift: secret argument
• Internal RPC protocol: you’re the
boss
• Less Doable
• SQL: one way ticket, also you’re
not percona
• memcache: not extensible so not
backwards compatible
35. t = start
nginx
cache python
db internet
the java
t = end
36. timing and structure
• Timing
• distributed = clock skew
• Structure -- two approaches
• Encode in ID
• Encode in back-pointers