SoftwareArchitectureusing ØMQbyPieter HintjensStrange Loop 2012Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
A complexstory is besttold as aseries ofvacuous 1-linersPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
90% ofsoftware istrash.90% of therest will betrash RSNPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
We basicallydont knowhow tomake codethat cansurvive ten,let alone 50yearsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter H...
The mostdifficultchallengein ourprofessionis simpleaccuracyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Future codehas to talkto code, hasto be chatty,sociable,well-connectedPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hint...
When wecan movefaster,where wego is morecritical thanever.Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Writingdistributedcode is likea live jamsession.Its all aboutother peoplePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter H...
How weconnect toeach othermattersmore thanwho we arePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
The physicsof softwareis thephysics ofpeoplePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Ideas arecheap.Execution isthe hardpartPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Makingperfectsoftware iseasy, onceyou learnthe trick(which iskinda hard)Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hi...
SimplicityalwaysbeatsfunctionalityPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Problemsare not allequal, andmost areillusionsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
When youknow thereal problemyou havedone halfthe workPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Do nothingthat is not aminimal,plausibleanswer to awell-definedproblemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hint...
Everycommitshould beshippablePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Design byremovingproblems,not addingfeaturesPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Five Stepsto Satori:Learn,Draw,Divide,Conquer,RepeatPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
1.Learn thelanguagebefore youwrite apoemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
2.If it lookspretty, itsmore likelyto workPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
3.A goodcontract isworth athousandsassumptionsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
4.When youtake smallsteps, ithurts lesswhen youfallPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
5.Solve oneproblem,and repeatuntil you runout of timeor moneyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Distributedsoftwarelives or diesby itsprotocolsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Protocolsare contractsthat describethe rightsandobligationsof each partyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hi...
An unprotocoltakes minutesto explain,hours todesign, days towrite, weeks toprove, monthsto mature, andyears toreplacePhoto...
Use human                         nom-protocol =                                     open-peeringlanguage                 ...
Use GPLv3for youropen specs.Remixabilityis freedomPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
If yourewilling togive upflexibility forspeed youdeserveneitherflexibility norspeedPhotos by Pieter Hintjenscc-by-sa © 201...
Use cheaptext for thelow-volumechattycontrolcommandsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Use nastyhand-codedbinary forthe high-volume dataPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
ØMQframingmakes alousy codecbut a greatseparatorPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
A hand-craftedcodec canalways beata genericserializerPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
A code-generatedcodec canalways beata hand-crafted onePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
iMatix GSL:technologyso dangerouswe had tolock it up foryearsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
File transferis thezombieproblem ofdistributedapplicationsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Routersockets arethe beatingheart ofevery realØMQprotocolenginePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
The worldneeds achunked,flow-controlled,restartable,cancellable,async,multicastfile transferØMQ protocolPhotos by Pieter H...
No matter                         C:   fetchhow hard                          S:   chunk 1you push, a                     ...
That                              C:   fetch chunk 1annoying                          S:   send chunk 1pause after        ...
You can,                          C:   fetch chunk 1and Ive                          C:   fetch chunk 2tested this,       ...
Request-                          C:   subscribereply is just                     C:   send credita vulgar                ...
On a routersocket, youshouldnever hit thehigh-watermarkPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Heartbeatsare ourprotocolsway ofasking if westill carePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Protocol                          command_tstack                              *request ==                                 ...
Statemachinesare a perfectdomainlanguage forprotocolenginesPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Statemachinescan be cudlyand gentle,when youget to knowthemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
You dontwant to betagainst acompilerPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
If youre notthinking ofsecurity,security isprobablythinking ofyouPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
Forconnectedbidirectionalprotocolsover ØMQuse SASLPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
For looselyconnectedand one-wayprotocolsover ØMQ,use AES andsuchPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
secure-nom = open-peeringSASL over                                     *use-peeringØMQ is                            open-...
Theory isfine intheory, butin practice,practice isbetterPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
FileMQ is afile sharingprotocol andstack overØMQ.Reusableuntil 2062Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
1. Aim for 50 yearsTo sum it up:                     2. Its all about people                                  3. Minimal p...
Upcoming SlideShare
Loading in...5
×

Software Architecture over ZeroMQ

2,842

Published on

Presented at Strange Loop 2012
How to build large systems that can survive 50 years.

Software Architecture over ZeroMQ

  1. 1. SoftwareArchitectureusing ØMQbyPieter HintjensStrange Loop 2012Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  2. 2. A complexstory is besttold as aseries ofvacuous 1-linersPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  3. 3. 90% ofsoftware istrash.90% of therest will betrash RSNPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  4. 4. We basicallydont knowhow tomake codethat cansurvive ten,let alone 50yearsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  5. 5. The mostdifficultchallengein ourprofessionis simpleaccuracyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  6. 6. Future codehas to talkto code, hasto be chatty,sociable,well-connectedPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  7. 7. When wecan movefaster,where wego is morecritical thanever.Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  8. 8. Writingdistributedcode is likea live jamsession.Its all aboutother peoplePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  9. 9. How weconnect toeach othermattersmore thanwho we arePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  10. 10. The physicsof softwareis thephysics ofpeoplePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  11. 11. Ideas arecheap.Execution isthe hardpartPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  12. 12. Makingperfectsoftware iseasy, onceyou learnthe trick(which iskinda hard)Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  13. 13. SimplicityalwaysbeatsfunctionalityPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  14. 14. Problemsare not allequal, andmost areillusionsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  15. 15. When youknow thereal problemyou havedone halfthe workPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  16. 16. Do nothingthat is not aminimal,plausibleanswer to awell-definedproblemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  17. 17. Everycommitshould beshippablePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  18. 18. Design byremovingproblems,not addingfeaturesPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  19. 19. Five Stepsto Satori:Learn,Draw,Divide,Conquer,RepeatPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  20. 20. 1.Learn thelanguagebefore youwrite apoemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  21. 21. 2.If it lookspretty, itsmore likelyto workPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  22. 22. 3.A goodcontract isworth athousandsassumptionsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  23. 23. 4.When youtake smallsteps, ithurts lesswhen youfallPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  24. 24. 5.Solve oneproblem,and repeatuntil you runout of timeor moneyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  25. 25. Distributedsoftwarelives or diesby itsprotocolsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  26. 26. Protocolsare contractsthat describethe rightsandobligationsof each partyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  27. 27. An unprotocoltakes minutesto explain,hours todesign, days towrite, weeks toprove, monthsto mature, andyears toreplacePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  28. 28. Use human nom-protocol = open-peeringlanguage *use-peeringin your open-peering =unprotocols. C:OHAI ( S:OHAI-OK / S:WTF ) use-peering =ORLY? C:ICANHAZYARLY! / S:CHEEZBURGER / C:HUGZ S:HUGZ-OK / S:HUGZ C:HUGZ-OKPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  29. 29. Use GPLv3for youropen specs.Remixabilityis freedomPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  30. 30. If yourewilling togive upflexibility forspeed youdeserveneitherflexibility norspeedPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  31. 31. Use cheaptext for thelow-volumechattycontrolcommandsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  32. 32. Use nastyhand-codedbinary forthe high-volume dataPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  33. 33. ØMQframingmakes alousy codecbut a greatseparatorPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  34. 34. A hand-craftedcodec canalways beata genericserializerPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  35. 35. A code-generatedcodec canalways beata hand-crafted onePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  36. 36. iMatix GSL:technologyso dangerouswe had tolock it up foryearsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  37. 37. File transferis thezombieproblem ofdistributedapplicationsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  38. 38. Routersockets arethe beatingheart ofevery realØMQprotocolenginePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  39. 39. The worldneeds achunked,flow-controlled,restartable,cancellable,async,multicastfile transferØMQ protocolPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  40. 40. No matter C: fetchhow hard S: chunk 1you push, a S: chunk 2file will not S: chunk 3just go downa socketPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  41. 41. That C: fetch chunk 1annoying S: send chunk 1pause after C: fetch chunk 2 S: send chunk 2you finish C: fetch chunk 3your beer, S: send chunk 3before you C: fetch chunk 4catch thewaiters eyePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  42. 42. You can, C: fetch chunk 1and Ive C: fetch chunk 2tested this, C: fetch chunk 3 S: send chunk 1order a new C: fetch chunk 4beer before S: send chunk 2your old one S: send chunk 3is emptyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  43. 43. Request- C: subscribereply is just C: send credita vulgar S: send chunk S: send chunksubclass of C: send creditpublish- S: send chunksubscribePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  44. 44. On a routersocket, youshouldnever hit thehigh-watermarkPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  45. 45. Heartbeatsare ourprotocolsway ofasking if westill carePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  46. 46. Protocol command_tstack *request == command_decode (socket)messagecodec execute_engine+ (command)protocolenginePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  47. 47. Statemachinesare a perfectdomainlanguage forprotocolenginesPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  48. 48. Statemachinescan be cudlyand gentle,when youget to knowthemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  49. 49. You dontwant to betagainst acompilerPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  50. 50. If youre notthinking ofsecurity,security isprobablythinking ofyouPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  51. 51. Forconnectedbidirectionalprotocolsover ØMQuse SASLPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  52. 52. For looselyconnectedand one-wayprotocolsover ØMQ,use AES andsuchPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  53. 53. secure-nom = open-peeringSASL over *use-peeringØMQ is open-peering = C:OHAIdarned *( S:ORLY C:YARLY )simple ( S:OHAI-OK / S:WTF ) ORLY = 1*mechanism challenge mechanism = string challenge = *OCTET YARLY = mechanism response response = *OCTETPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  54. 54. Theory isfine intheory, butin practice,practice isbetterPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  55. 55. FileMQ is afile sharingprotocol andstack overØMQ.Reusableuntil 2062Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  56. 56. 1. Aim for 50 yearsTo sum it up: 2. Its all about people 3. Minimal plausible solutionszero.mq/ch6 4. To real immediate problems 5. Document the contracts 6. Cheap and Nasty codecsThe Weird 7. Code generation rocksFish Book, 8. Router sockets rockcoming soon 9. CBFC > HWMfrom OReilly 10. Learn state machines 11. Learn about SASLPhotos by Pieter Hintjens 12. Worked example: FileMQcc-by-sa © 2012 Pieter Hintjens
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×