CRAFT 2016

1. Microservice
Protocols of Interaction
Todd L. Montgomery
@toddlmontgomery

2. About me…

3. What is a Protocol?
Why should we care!?

4. @toddlmontgomery
pro·to·col noun ˈprō-tə-ˌkȯl, -ˌkōl, -ˌkäl, -kəl
...
3 b : a set of conventions governing the treatment and especially the
formatting of data in an electronic communications
system <network protocols>
...
3 a : a code prescribing strict adherence to correct etiquette
and precedence (as in diplomatic exchange and in the military
services) <a breach of protocol>

5. Protocols of Interaction
Wire Protocol, Method Calls, Shared Memory
Interactions, etc.

6. Microservice Architectures

7. Forced Decoupling
via an
“Asynchronous, Binary Boundary”

8. Forced Loose Coupling

9. The truth is…
Protocols can and do Couple

10. Protocols of Interaction
are quite important!
Protocols of Interaction Matter!

11. The Environment

12. Networks, and especially the Internet,
are Hostile Environments

14. Data can be
lost,
duplicated, and
re-ordered!!

15. TCP connections can…
be closed
unexpectedly
end in an
unknown state
be intercepted
by idiots, er Proxies

16. Duplicated
Re-Ordered
Lost
Which means
Data over TCP* might be…
* - When connections are re-established

17. Don’t assume the network
is reliable
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

18. Case Studies

19. Case Study 1
Loose Ordering

20. @toddlmontgomery
Sync
Requests
&
Responses
Request
Request
Request
Response
Response
Response
Throughput limited by
Round-Trip Time (RTT)!

21. @toddlmontgomery
Async
Requests
&
Responses
Request
Request
Request
Response
Response
Response
Throughput less limited by
Round-Trip Time!

22. @toddlmontgomery
Async
Requests
&
Responses
Correlation!
Request 0
Request 1
Request 2
Response 0
Response 1
Response 2

23. Aside…

24. Ordering is an Illusion!!

25. Compiler can re-order
Runtime can re-order
CPU can re-order

26. Ordering has to be imposed!

27. @toddlmontgomery
Async
Requests
&
Responses
Correlation!
Request 0
Request 1
Request 2
Response 0
Response 1
Response 2

28. @toddlmontgomery
Correlation!
Request 0
Request 1
Request 2
Response 0
Response 1
Response 2
Ordering

29. @toddlmontgomery
Correlation!
Request 0
Request 1
Request 2
Response 0
Response 1
Response 2
(Valid)
Re-Ordering
(one of many)

30. @toddlmontgomery
Handling the Unexpected
Request 0
Response 1
Invalid, Drop We only know of 0.
1 is unknown!

31. SCTP
HTTP/2 (SPDY)
…
most OSI Layer 4 protocols

32. Case Study 2
Can you hear me now?
Timeouts & Retries

33. @toddlmontgomery
Request
ACK
Processing
Handling the unexpected

34. @toddlmontgomery
Request
TimeoutInterval
X

35. @toddlmontgomery
Request
ACK
Processing
XTimeoutInterval
Retransmit at end of interval

36. @toddlmontgomery
ACK
Processing
…
Avoid Spurious Retransmits
Retransmit
Original
TimeoutInterval

37. @toddlmontgomery
Interval =
N x “typical” RTT
Account for processing delay
X
TimeoutInterval
“Average”

38. @toddlmontgomery
Measure! But very “noisy”?
RTTMeasurement
Variances in
processing,
transmission,
etc.

39. TCP Retransmit Timeout (RTO)
Err = M - A
A <- A + gErr
D <- D + h(|Err| - D)
RTO = A + 4D
M = measurement, A = smoothed average,
D = smoothed mean deviation,
g and h = gain constants (0 to 1)

40. TCP Retransmit Timeout (RTO)
Err = M - A
A <- A + gErr
D <- D + h(|Err| - D)
RTO = A + 4D
Do you measure on a Retransmit? NO!

41. @toddlmontgomery
Does processing twice hurt?
X
Original
ACK
Retrans
Process Once
Process Twice
TimeoutInterval

42. @toddlmontgomery
Are Original & Retransmit treated the same?
X
Original
ACK
Retrans
Process Once
Process Twice
TimeoutInterval

43. TCP
SCTP
Aeron
…
anything with reliability

44. Case Study 3
What I Need! When I Need It!
“Lifetime” Management

45. “Managing” Application
Working Set
or
Service Liveness

46. Caching Algorithms
LRU, MRU, PLRU, RR,
SLRU, LFU, …
“Liveness” is essential

48. @toddlmontgomery
Request
ACK
Service A
is Alive!
Service B
is Alive!
Service A Service B
Consequence of Processing

49. @toddlmontgomery
Keepalive
Keepalive
Service A
is Alive!
Service B
is Alive!
Service A Service B
Absence of Processing

50. RIP Route Deletion
Step 0 - route info broadcast @30 seconds
Step 1 (3 min) - Set Distance to Infinity (16)
Step 2 (+1 min) - Delete Route
Aside… RIP… aptly named

51. Aeron Driver Keepalive
Time of Last Activity = Shared Variable
Doesn’t need to be a message…

52. @toddlmontgomery
Bye
Bye
Service A
is gone!
Service B
is gone!
Service A Service B
Optimization, but insufficient with arbitrary failures

53. Liveness often exists across
transient connectivity

54. So…
Don’t conflate transport
state with liveness!
Like TCP connection state

55. Dead TCP connection
!=
Dead Service

56. Live TCP connection
!=
Live Service

57. BGP
OSPF
Transports
…
almost every protocol

58. Case Study 4
Elasti-What?
Self-Similar Behavior

59. Request
X
Request
X
Request
X
Request
X, X, X
Multiple same/similar requests at the same time
Response
X, X, X

60. Similar Problem…
Reliable Multicast

61. 1, 2, 3
1, 2, 3 1, 2, 3 1, 2, 3
Non-correlated loss
X X X

62. NAK
1, 2, 3
NAK
2
NAK
1
NAK
3
Request individual lost data
Retransmit
1, 2, 3

63. 1, 2, 3
1, 2, 3 1, 2, 3 1, 2, 3
Temporally/Spatially Correlated Loss
X X X

64. NAK
2
NAK
2
NAK
2
NAK
2, 2, 2
Multiple requests for same data
Retransmit
2, 2, 2

65. Request
2
Request
2
Request
2
Request
2, 2, 2
It’s a generic problem!

66. Request
2
Request
2
Request
2
Request
2, 2, 2
Overloading Responder & Network

68. Request
2
Publish Requests
Don’t Immediately Request, Listen first
Timeout!
Request
2
Request
2
Suppress
Request

69. Request
2
How long to wait & listen for?
Timeout!
Request
2
Request
2
Suppress
Request

70. Statistics to the Rescue!

71. SRM Backoff
RandomBackoff = [C1, C1+C2] * 1-way delay
Random is more than good enough

72. Request
2
Request
2
Request
2, 2
Must also shed duplicates on the responder
Response
2, 2
Shed second
“Request 2”
if too soon
X
X

73. SRM
PGM
Aeron
…
http://en.wikipedia.org/wiki/Scalable_Reliable_Multicast
http://www.eurecom.fr/en/publication/107/detail/optimal-multicast-feedback

74. Case Study 5
Hey, Slow Down!
Flow (& Congestion) Control

76. @toddlmontgomery
Data
Data
Data
ACK
ACK
ACK
Throughput = Data Length / RTT
RTT
Stop-And-Wait
Flow Control

77. Delay
Bandwidth
BDP = (Byte / sec) * sec = Bytes
BDP
(Buffer)

78. @toddlmontgomery
Data
ACK
RTT
Throughput = N * Data Length / RTT
… N Data
“Blobs”

79. So…
How big is N?
This is surprisingly hard to answer

80. It depends…

81. Big… but
Don’t overflow receiver
Don’t overflow “network”

82. TCP Flow Control
Receiver advertises N

83. TCP Congestion Control
Sender probes for network N

84. TCP Sender
min(Receiver N, Network N)
Only go as fast as Network & Receiver

85. ReactiveStreams
Subscriber uses explicit request(N)
Publisher assumes best case
http://www.reactive-streams.org/

86. Takeaways!

87. Protocols of interaction are important &
can be tremendously impactful
for better or worse…

88. @toddlmontgomery
Questions?
• IETF http://www.ietf.org/
• Aeron https://github.com/real-logic/Aeron
• Twitter @toddlmontgomery
Thank You!