Software Architecture over ZeroMQ
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Software Architecture over ZeroMQ

on

  • 2,261 views

Presented at Strange Loop 2012

Presented at Strange Loop 2012
How to build large systems that can survive 50 years.

Statistics

Views

Total Views
2,261
Views on SlideShare
2,259
Embed Views
2

Actions

Likes
3
Downloads
68
Comments
0

2 Embeds 2

https://twitter.com 1
http://www.slashdocs.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Software Architecture over ZeroMQ Presentation Transcript

  • 1. SoftwareArchitectureusing ØMQbyPieter HintjensStrange Loop 2012Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 2. A complexstory is besttold as aseries ofvacuous 1-linersPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 3. 90% ofsoftware istrash.90% of therest will betrash RSNPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 4. We basicallydont knowhow tomake codethat cansurvive ten,let alone 50yearsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 5. The mostdifficultchallengein ourprofessionis simpleaccuracyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 6. Future codehas to talkto code, hasto be chatty,sociable,well-connectedPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 7. When wecan movefaster,where wego is morecritical thanever.Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 8. Writingdistributedcode is likea live jamsession.Its all aboutother peoplePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 9. How weconnect toeach othermattersmore thanwho we arePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 10. The physicsof softwareis thephysics ofpeoplePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 11. Ideas arecheap.Execution isthe hardpartPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 12. Makingperfectsoftware iseasy, onceyou learnthe trick(which iskinda hard)Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 13. SimplicityalwaysbeatsfunctionalityPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 14. Problemsare not allequal, andmost areillusionsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 15. When youknow thereal problemyou havedone halfthe workPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 16. Do nothingthat is not aminimal,plausibleanswer to awell-definedproblemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 17. Everycommitshould beshippablePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 18. Design byremovingproblems,not addingfeaturesPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 19. Five Stepsto Satori:Learn,Draw,Divide,Conquer,RepeatPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 20. 1.Learn thelanguagebefore youwrite apoemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 21. 2.If it lookspretty, itsmore likelyto workPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 22. 3.A goodcontract isworth athousandsassumptionsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 23. 4.When youtake smallsteps, ithurts lesswhen youfallPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 24. 5.Solve oneproblem,and repeatuntil you runout of timeor moneyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 25. Distributedsoftwarelives or diesby itsprotocolsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 26. Protocolsare contractsthat describethe rightsandobligationsof each partyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 27. An unprotocoltakes minutesto explain,hours todesign, days towrite, weeks toprove, monthsto mature, andyears toreplacePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 28. Use human nom-protocol = open-peeringlanguage *use-peeringin your open-peering =unprotocols. C:OHAI ( S:OHAI-OK / S:WTF ) use-peering =ORLY? C:ICANHAZYARLY! / S:CHEEZBURGER / C:HUGZ S:HUGZ-OK / S:HUGZ C:HUGZ-OKPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 29. Use GPLv3for youropen specs.Remixabilityis freedomPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 30. If yourewilling togive upflexibility forspeed youdeserveneitherflexibility norspeedPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 31. Use cheaptext for thelow-volumechattycontrolcommandsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 32. Use nastyhand-codedbinary forthe high-volume dataPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 33. ØMQframingmakes alousy codecbut a greatseparatorPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 34. A hand-craftedcodec canalways beata genericserializerPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 35. A code-generatedcodec canalways beata hand-crafted onePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 36. iMatix GSL:technologyso dangerouswe had tolock it up foryearsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 37. File transferis thezombieproblem ofdistributedapplicationsPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 38. Routersockets arethe beatingheart ofevery realØMQprotocolenginePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 39. The worldneeds achunked,flow-controlled,restartable,cancellable,async,multicastfile transferØMQ protocolPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 40. No matter C: fetchhow hard S: chunk 1you push, a S: chunk 2file will not S: chunk 3just go downa socketPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 41. That C: fetch chunk 1annoying S: send chunk 1pause after C: fetch chunk 2 S: send chunk 2you finish C: fetch chunk 3your beer, S: send chunk 3before you C: fetch chunk 4catch thewaiters eyePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 42. You can, C: fetch chunk 1and Ive C: fetch chunk 2tested this, C: fetch chunk 3 S: send chunk 1order a new C: fetch chunk 4beer before S: send chunk 2your old one S: send chunk 3is emptyPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 43. Request- C: subscribereply is just C: send credita vulgar S: send chunk S: send chunksubclass of C: send creditpublish- S: send chunksubscribePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 44. On a routersocket, youshouldnever hit thehigh-watermarkPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 45. Heartbeatsare ourprotocolsway ofasking if westill carePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 46. Protocol command_tstack *request == command_decode (socket)messagecodec execute_engine+ (command)protocolenginePhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 47. Statemachinesare a perfectdomainlanguage forprotocolenginesPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 48. Statemachinescan be cudlyand gentle,when youget to knowthemPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 49. You dontwant to betagainst acompilerPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 50. If youre notthinking ofsecurity,security isprobablythinking ofyouPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 51. Forconnectedbidirectionalprotocolsover ØMQuse SASLPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 52. For looselyconnectedand one-wayprotocolsover ØMQ,use AES andsuchPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 53. secure-nom = open-peeringSASL over *use-peeringØMQ is open-peering = C:OHAIdarned *( S:ORLY C:YARLY )simple ( S:OHAI-OK / S:WTF ) ORLY = 1*mechanism challenge mechanism = string challenge = *OCTET YARLY = mechanism response response = *OCTETPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 54. Theory isfine intheory, butin practice,practice isbetterPhotos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 55. FileMQ is afile sharingprotocol andstack overØMQ.Reusableuntil 2062Photos by Pieter Hintjenscc-by-sa © 2012 Pieter Hintjens
  • 56. 1. Aim for 50 yearsTo sum it up: 2. Its all about people 3. Minimal plausible solutionszero.mq/ch6 4. To real immediate problems 5. Document the contracts 6. Cheap and Nasty codecsThe Weird 7. Code generation rocksFish Book, 8. Router sockets rockcoming soon 9. CBFC > HWMfrom OReilly 10. Learn state machines 11. Learn about SASLPhotos by Pieter Hintjens 12. Worked example: FileMQcc-by-sa © 2012 Pieter Hintjens