Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1NAIWYK. Todd Montgomery describes some common problems that have arisen in protocol design, using examples such as HTTP/2, Aeron, etc., and how the solutions can be applied to microservices. Filmed at qconlondon.com. Todd Montgomery is a networking hacker who has researched, designed, and built numerous protocols, messaging-oriented middleware systems, and real-time data systems, done research for NASA, contributed to the IETF and IEEE, and cofounded two startups. He currently works for Kaazing as Chief Architect.

1. Protocols of Interaction
Best Current Practices
Todd L. Montgomery
@toddlmontgomery

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/protocols-microservices

3. Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. What is a Protocol?
Why should I care!?

5. @toddlmontgomery
pro·to·col noun ˈprō-tə-ˌkȯl, -ˌkōl, -ˌkäl, -kəl
...
3 b : a set of conventions governing the treatment and especially the
formatting of data in an electronic communications
system <network protocols>
...
3 a : a code prescribing strict adherence to correct etiquette
and precedence (as in diplomatic exchange and in the military
services) <a breach of protocol>

6. Protocols of Interaction Matter

7. In an emerging era of micro-services,
protocols of interaction matter
Protocols are a rich source of solutions
to micro-service problems

8. Algorithms
Performance
Concurrency
Security
Multi-Disciplinary
Number Theory
Statistics
Graph Theory
Biology ?!

9. Networks, and especially the Internet,
are Hostile Environments

11. Data can be
lost,
duplicated, and
re-ordered!!

12. TCP connections can…
be closed
unexpectedly
end in an
unknown state
be intercepted
by idiots, er Proxies

13. Duplicated
Re-Ordered
Lost
Which means
Data over TCP* might be…
* - When connections are re-established

14. Case Study 1
Loose Ordering = The New Normal
(De)multiplexing

15. @toddlmontgomery
Sync
Requests
&
Responses
Request
Request
Request
Response
Response
Response
Throughput limited by
Round-Trip Time (RTT)!

16. @toddlmontgomery
Async
Requests
&
Responses
Request
Request
Request
Response
Response
Response
Throughput less limited by
Round-Trip Time!

17. @toddlmontgomery
Async
Requests
&
Responses
Correlation!
Request 0
Request 1
Request 2
Response 0
Response 1
Response 2

18. Aside…

19. Ordering is an Illusion!!

20. Compiler can re-order
Runtime can re-order
CPU can re-order

21. Ordering has to be imposed!

22. @toddlmontgomery
Async
Requests
&
Responses
Correlation!
Request 0
Request 1
Request 2
Response 0
Response 1
Response 2

23. @toddlmontgomery
Correlation!
Request 0
Request 1
Request 2
Response 0
Response 1
Response 2
Ordering

24. @toddlmontgomery
Correlation!
Request 0
Request 1
Request 2
Response 0
Response 1
Response 2
(Valid)
Re-Ordering

25. @toddlmontgomery
Handling the Unexpected
Request 0
Response 1
Invalid, Drop We only know of 0.
1 is unknown!

26. SCTP
HTTP/2 (SPDY)
…
most OSI Layer 4 protocols

27. Case Study 2
Can you hear me now?
Timeouts & Retries

28. @toddlmontgomery
Request
ACK
Processing
Handling the unexpected

29. @toddlmontgomery
Request
TimeoutInterval
X

30. @toddlmontgomery
Request
ACK
Processing
XTimeoutInterval
Retransmit at end of interval

31. @toddlmontgomery
ACK
Processing
…
Spurious Retransmits
Retransmit
Original
TimeoutInterval

32. @toddlmontgomery
Interval =
N x “typical” RTT
Account for processing delay
X
TimeoutInterval
“Average”

33. @toddlmontgomery
Measure! But very “noisy”?
RTTMeasurement
Variances in
processing,
transmission,
etc.

34. TCP Retransmit Timeout (RTO)
Err = M - A
A <- A + gErr
D <- D + h(|Err| - D)
RTO = A + 4D
M = measurement, A = smoothed average,
D = smoothed mean deviation,
g and h = gain constants (0 to 1)

35. TCP Retransmit Timeout (RTO)
Err = M - A
A <- A + gErr
D <- D + h(|Err| - D)
RTO = A + 4D
Do you measure on a Retransmit? NO!

36. @toddlmontgomery
Does processing twice hurt?
X
Original
ACK
Retrans
Process Once
Process Twice
TimeoutInterval

37. @toddlmontgomery
Are Original & Retransmit treated the same?
X
Original
ACK
Retrans
Process Once
Process Twice
TimeoutInterval

38. TCP
SCTP
Aeron
…
anything with reliability

39. Case Study 3
What I Need! When I Need It!
“Lifetime” Management

40. “Managing” Application
Working Set

41. Caching Algorithms
LRU, MRU, PLRU, RR,
SLRU, LFU, …
“Liveness” is essential

43. @toddlmontgomery
Request
ACK
Service A
is Alive!
Service B
is Alive!
Service A Service B
Consequence of Processing

44. @toddlmontgomery
Keepalive
Keepalive
Service A
is Alive!
Service B
is Alive!
Service A Service B
Absence of Processing

45. RIP Route Deletion
Step 0 - route info broadcast @30 seconds
Step 1 (3 min) - Set Distance to Infinity (16)
Step 2 (+1 min) - Delete Route
Aside… RIP… aptly named

46. Aeron Driver Keepalive
Time of Last Activity = Shared Variable
Doesn’t need to be a message…

47. @toddlmontgomery
Bye
Bye
Service A
is gone!
Service B
is gone!
Service A Service B
Optimization, but insufficient with arbitrary failures

48. Liveness often exists across
transient connectivity

49. So…
Don’t conflate transport
state with liveness!
Like TCP connection state

50. BGP
OSPF
Transports
…
almost every protocol

51. Case Study 4
Elasti-What?
Self-Similar Behavior

52. Request
X
Request
X
Request
X
Request
X, X, X
Multiple same/similar requests at the same time
Response
X, X, X

53. Similar Problem…
Reliable Multicast

54. 1, 2, 3
1, 2, 3 1, 2, 3 1, 2, 3
Non-correlated loss
X X X

55. NAK
1, 2, 3
NAK
2
NAK
1
NAK
3
Request individual lost data
Retransmit
1, 2, 3

56. 1, 2, 3
1, 2, 3 1, 2, 3 1, 2, 3
Temporally/Spatially Correlated Loss
X X X

57. NAK
2
NAK
2
NAK
2
NAK
2, 2, 2
Multiple requests for same data
Retransmit
2, 2, 2

58. Request
2
Request
2
Request
2
Request
2, 2, 2
It’s a generic problem!

59. Request
2
Request
2
Request
2
Request
2, 2, 2
Overloading Responder & Network

61. Request
2
Don’t Immediately Request, Listen first
Timeout!
Request
2
Request
2
Suppress
Request

62. Request
2
How long to wait & listen for?
Timeout!
Request
2
Request
2
Suppress
Request

63. Statistics to the Rescue!

64. SRM Backoff
RandomBackoff = [C1, C1+C2] * 1-way delay
Random is more than good enough

65. Optimal Multicast Feedback
double RandomBackoff(double T_maxBackoff, double groupSize)
{
double lambda = log(groupSize) + 1;
double x = UniformRand(lambda/T_maxBackoff) +
lambda / (T_maxBackoff*(exp(lambda)-1));
return ((T_maxBackoff/lambda) *
log(x*(exp(lambda)-1)*(T_maxBackoff/lambda)));
}
Truncated Exponential Distribution

66. Request
2
Request
2
Request
2, 2
Must also shed duplicates on the responder
Response
2, 2
Shed second
“Request 2”
if too soon
X
X

67. SRM
PGM
Aeron
…
http://en.wikipedia.org/wiki/Scalable_Reliable_Multicast
http://www.eurecom.fr/en/publication/107/detail/optimal-multicast-feedback

68. Case Study 5
Hey, Slow Down!
Flow (& Congestion) Control

70. @toddlmontgomery
Data
Data
Data
ACK
ACK
ACK
Throughput = Data Length / RTT
RTT
Stop-And-Wait
Flow Control

71. Delay
Bandwidth
BDP = (Byte / sec) * sec = Bytes
BDP
(Buffer)

72. @toddlmontgomery
Data
ACK
RTT
Throughput = N * Data Length / RTT
… N Data
“Blobs”

73. So…
How big is N?
This is surprisingly hard to answer

74. It depends…

75. Big… but
Don’t overflow receiver
Don’t overflow “network”

76. TCP Flow Control
Receiver advertises N

77. TCP Congestion Control
Sender probes for network N

78. TCP Sender
min(Receiver N, Network N)
Only go as fast as Network & Receiver

79. TCP
Aeron
…
http://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm

80. One more thing…

81. Queue Management
Perhaps the single most useful thing!

82. Effective management of
queues can not be overlooked

83. Unbounded Queues are bad, m’kay

84. Bounding implies
Back pressure and/or Dropping

85. CoDel
locally minimize delay in queue
combat bufferbloat
http://en.wikipedia.org/wiki/CoDel

86. Just a taste…

87. Takeaways!

88. Protocols are a rich source of
solutions to complicated problems
Protocols of interaction matter & can
be tremendously impactful for
better or worse…

89. @toddlmontgomery
Questions?
• IETF http://www.ietf.org/
• Aeron https://github.com/real-logic/Aeron
• SlideShare http://www.slideshare.com/toddleemontgomery
• Twitter @toddlmontgomery
Thank You!

90. Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/protocols-
microservices