Your SlideShare is downloading. ×
Software Architecture over ZeroMQ
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Software Architecture over ZeroMQ

2,368
views

Published on

Presented at Strange Loop 2012 …

Presented at Strange Loop 2012
How to build large systems that can survive 50 years.


0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,368
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
107
Comments
0
Likes
6
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. SoftwareArchitectureusing ØMQbyPieter HintjensStrange Loop 2012Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 2. A complexstory is besttold as aseries ofvacuous 1-linersPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 3. 90% ofsoftware istrash.90% of therest will betrash RSNPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 4. We basicallydont knowhow tomake codethat cansurvive ten,let alone 50yearsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 5. The mostdifficultchallengein ourprofessionis simpleaccuracyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 6. Future codehas to talkto code, hasto be chatty,sociable,well-connectedPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 7. When wecan movefaster,where wego is morecritical thanever.Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 8. Writingdistributedcode is likea live jamsession.Its all aboutother peoplePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 9. How weconnect toeach othermattersmore thanwho we arePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 10. The physicsof softwareis thephysics ofpeoplePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 11. Ideas arecheap.Execution isthe hardpartPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 12. Makingperfectsoftware iseasy, onceyou learnthe trick(which iskinda hard)Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 13. SimplicityalwaysbeatsfunctionalityPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 14. Problemsare not allequal, andmost areillusionsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 15. When youknow thereal problemyou havedone halfthe workPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 16. Do nothingthat is not aminimal,plausibleanswer to awell-definedproblemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 17. Everycommitshould beshippablePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 18. Design byremovingproblems,not addingfeaturesPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 19. Five Stepsto Satori:Learn,Draw,Divide,Conquer,RepeatPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 20. 1.Learn thelanguagebefore youwrite apoemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 21. 2.If it lookspretty, itsmore likelyto workPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 22. 3.A goodcontract isworth athousandsassumptionsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 23. 4.When youtake smallsteps, ithurts lesswhen youfallPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 24. 5.Solve oneproblem,and repeatuntil you runout of timeor moneyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 25. Distributedsoftwarelives or diesby itsprotocolsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 26. Protocolsare contractsthat describethe rightsandobligationsof each partyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 27. An unprotocoltakes minutesto explain,hours todesign, days towrite, weeks toprove, monthsto mature, andyears toreplacePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 28. Use human nom-protocol = open-peeringlanguage *use-peeringin your open-peering =unprotocols. C:OHAI ( S:OHAI-OK / S:WTF ) use-peering =ORLY? C:ICANHAZYARLY! / S:CHEEZBURGER / C:HUGZ S:HUGZ-OK / S:HUGZ C:HUGZ-OKPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 29. Use GPLv3for youropen specs.Remixabilityis freedomPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 30. If yourewilling togive upflexibility forspeed youdeserveneitherflexibility norspeedPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 31. Use cheaptext for thelow-volumechattycontrolcommandsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 32. Use nastyhand-codedbinary forthe high-volume dataPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 33. ØMQframingmakes alousy codecbut a greatseparatorPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 34. A hand-craftedcodec canalways beata genericserializerPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 35. A code-generatedcodec canalways beata hand-crafted onePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 36. iMatix GSL:technologyso dangerouswe had tolock it up foryearsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 37. File transferis thezombieproblem ofdistributedapplicationsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 38. Routersockets arethe beatingheart ofevery realØMQprotocolenginePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 39. The worldneeds achunked,flow-controlled,restartable,cancellable,async,multicastfile transferØMQ protocolPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 40. No matter C: fetchhow hard S: chunk 1you push, a S: chunk 2file will not S: chunk 3just go downa socketPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 41. That C: fetch chunk 1annoying S: send chunk 1pause after C: fetch chunk 2 S: send chunk 2you finish C: fetch chunk 3your beer, S: send chunk 3before you C: fetch chunk 4catch thewaiters eyePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 42. You can, C: fetch chunk 1and Ive C: fetch chunk 2tested this, C: fetch chunk 3 S: send chunk 1order a new C: fetch chunk 4beer before S: send chunk 2your old one S: send chunk 3is emptyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 43. Request- C: subscribereply is just C: send credita vulgar S: send chunk S: send chunksubclass of C: send creditpublish- S: send chunksubscribePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 44. On a routersocket, youshouldnever hit thehigh-watermarkPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 45. Heartbeatsare ourprotocolsway ofasking if westill carePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 46. Protocol command_tstack *request == command_decode (socket)messagecodec execute_engine+ (command)protocolenginePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 47. Statemachinesare a perfectdomainlanguage forprotocolenginesPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 48. Statemachinescan be cudlyand gentle,when youget to knowthemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 49. You dontwant to betagainst acompilerPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 50. If youre notthinking ofsecurity,security isprobablythinking ofyouPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 51. Forconnectedbidirectionalprotocolsover ØMQuse SASLPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 52. For looselyconnectedand one-wayprotocolsover ØMQ,use AES andsuchPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 53. secure-nom = open-peeringSASL over *use-peeringØMQ is open-peering = C:OHAIdarned *( S:ORLY C:YARLY )simple ( S:OHAI-OK / S:WTF ) ORLY = 1*mechanism challenge mechanism = string challenge = *OCTET YARLY = mechanism response response = *OCTETPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 54. Theory isfine intheory, butin practice,practice isbetterPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 55. FileMQ is afile sharingprotocol andstack overØMQ.Reusableuntil 2062Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 56. 1. Aim for 50 yearsTo sum it up: 2. Its all about people 3. Minimal plausible solutionszero.mq/ch6 4. To real immediate problems 5. Document the contracts 6. Cheap and Nasty codecsThe Weird 7. Code generation rocksFish Book, 8. Router sockets rockcoming soon 9. CBFC > HWMfrom OReilly 10. Learn state machines 11. Learn about SASLPhotos by Pieter Hintjens 12. Worked example: FileMQcc-by-sa © 2012 Pieter Hintjens