Distributed systems collaborate to achieve collective goals via a system of rules. Rules that affords good hygiene, fault tolerance, effective communication and trusted feedback. These rules form protocols which enable the system to achieve its goals. Distributed and concurrent systems can be considered a social group that collaborates to achieve collective goals. In order to collaborate a system of rules must be applied, that affords good hygiene, fault tolerance, and effective communication to coordinate, share knowledge, and provide feedback in a polite trusted manner. These rules form a number of protocols which enable the group to act as a system which is greater than the sum of the individual components. In this talk, we will explore the history of protocols and their application when building distributed systems.

1. Interaction Protocols
Martin Thompson - @mjpt777

2. Code

5. Are protocols the most
significant human discovery?

6. protocol
noun ˈprō-tə-ˌkȯl
Source: http://www.merriam-webster.com/

7. protocol
noun ˈprō-tə-ˌkȯl
: a code prescribing strict adherence to
correct etiquette and precedence
Source: http://www.merriam-webster.com/

8. protocol
noun ˈprō-tə-ˌkȯl
: a code prescribing strict adherence to
correct etiquette and precedence
: a set of conventions governing the
treatment and formatting of data in an
electronic communications system
Source: http://www.merriam-webster.com/

9. Evolutionary Biology

10. Facial Expressions

11. Etiquette & Manners

12. Hygiene
Evolution of Communities
Disease<=>

13. Hygiene
Courtesy
Evolution of Communities
Disease
Society
<=>
<=>

14. Hygiene
Courtesy
Norms
Evolution of Communities
Disease
Society
Trust
<=>
<=>
<=>

15. “What is acceptable”
Rules of Engagement

16. “What is acceptable”
“Good conditions to succeed”
Rules of Engagement

17. “What is acceptable”
“Good conditions to succeed”
“Jus ad bellum, Jus ad bello”
Rules of Engagement

18. Concurrent & Distributed
Systems

19. How do components Interact?

20. IETF
(Internet Engineering Task Force)

22. 418
“I'm a teapot”

27. How should we document our
protocols?

28. API vs Protocol

29. Open, *[Read | Write], Close

30. Open, *[Read | Write], Close
1. Open: ...
2. Read: ...
3. Write: ...
4. Close: ...

31. Think events and then consider:
pre, post, and invariant
conditions

32. “What can possibly go wrong?”

34. Multicast Example

35. ACK / NAK Implosion

38. Where should we focus?

39. Many Aspects to Consider
• Layering
• Versioning
• Encoding
• Addressing
• Error Handling
• Flow Control
• Congestion Control
• Feedback
• Sequencing
• Batching
• Sync/Async
• Validation
• Trust
• Privacy

40. Who cares about waste?

41. Encoding
Don’t use text codecs!
Please use binary codecs

42. Encoding
“…but it’s human readable…”

43. 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-------------------------------------------------------------+
|R| Frame Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-------------------------------+
| Version |B|E| Flags | Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-------------------------------+
|R| Term Offset |
+-+-------------------------------------------------------------+
| Session ID |
+---------------------------------------------------------------+
| Stream ID |
+---------------------------------------------------------------+
| Term ID |
+---------------------------------------------------------------+
| Encoded Message ...
... |
+---------------------------------------------------------------+

44. Versioning

45. Versioning/Identity
• Protocols: What conversation?

46. Versioning/Identity
• Protocols: What conversation?
• Messages: What encoding?

47. Versioning/Identity
• Protocols: What conversation?
• Messages: What encoding?
• State: What instance?

49. MPMC Queue
http://www.1024cores.net/

50. Sync vs Async

51. Synchronous

52. Synchronous

53. Synchronous

54. Synchronous

55. Synchronous

56. Synchronous

57. Synchronous

58. Synchronous

59. Synchronous

60. Synchronous

61. Synchronous

62. Synchronous

63. Synchronous
What’s Happening?

64. Asynchronous

65. Asynchronous

66. Asynchronous

67. Asynchronous

68. Asynchronous

69. Asynchronous

70. Asynchronous

71. Asynchronous

72. Asynchronous

73. Asynchronous

74. Asynchronous

75. Asynchronous

76. Asynchronous

77. Asynchronous

78. Asynchronous
What’s Happening?

79. “But, but, synchronous
is so much easier...”

80. ...it’s all about managing state
via feedback

81. Events model the real world,
i.e. it’s distributed,
in space and time

82. When designing an API,
wrap Async with Sync
if you must

83. RPC / HTTP / TCP

84. TCP → TCP Fast Open → QUIC
HTTP/1.1 → SPDY → HTTP/2
TLS 1.2 → TLS 1.3

86. 0-RTT and replay attacks

87. Batching

88. Etiquette of a request

89. 100 GigE ?

90. Syscalls

91. sndmmsg, rcvmmsg()
Onload, VMA
DPDK, ef_vi, RDMA Verbs
Future of Syscalls

92. Natural Batching &
Mechanical Sympathy

93. ORMs vs SQL

94. Snake Oil Protocols

95. 2PC / XA

97. “Two-Phase Commit is not fault
tolerant because it uses a single
coordinator whose failure can
cause the protocol to block”

98. Guaranteed Delivery™

99. Applications should have
feedback & recovery protocols

100. Protocol Layering

101. Protocol Layering
Depend on behaviour,
not implementation

102. Know and take your
responsibilities seriously

103. Wrapping up…

104. Are protocols the most
significant human discovery?

105. Question, Hypothesis,
Prediction, Experiment,
Analysis

107. Why are protocols not
studied and practiced more?

108. Your next project =>
1. Frameworks? or Protocols?
2. ...

109. Code of Conduct
The Golden Rule

110. @mjpt777
https://github.com/real-logic/aeron
“A distributed system is one in which the failure
of a computer you didn't even know existed
can render your own computer unusable.”
- Leslie Lamport
Questions?