IN THE BACK OF YOUR MIND  beanstalkd and 0MQ through the Dominic System
THE DOMINIC SYSTEM
LEARN HOW TO IMPROVE YOUR MINDA book I am currently reading
MEMORY TECHNIQUES
MEMORY TECHNIQUES• The   book includes various techniques for improving memory
MEMORY TECHNIQUES• The   book includes various techniques for improving memory• Ithelps to translate abstract data (the br...
MEMORY TECHNIQUES• The    book includes various techniques for improving memory• Ithelps to translate abstract data (the b...
SINGLE DIGITS
SINGLE DIGITS• Just     count through the alphabet  • But   replace F with S  • And     skip to N when you    hit 9
SINGLE DIGITS                                 1   A                                 2   B• Just     count through the     ...
DOUBLE DIGITS
DOUBLE DIGITS• Convert   digit pairs into letters • 15   = AE, 23 = BC, 27 = BG, 39 = CN, 80 = HO, 80 = HA
DOUBLE DIGITS• Convert    digit pairs into letters • 15   = AE, 23 = BC, 27 = BG, 39 = CN, 80 = HO, 80 = HA• Use   the let...
DOUBLE DIGITS• Convert    digit pairs into letters • 15   = AE, 23 = BC, 27 = BG, 39 = CN, 80 = HO, 80 = HA• Use   the let...
IMAGERY IS KEY
IMAGERY IS KEY• Each   digit pair should create an image in your mind
IMAGERY IS KEY• Each   digit pair should create an image in your mind• The   image need to be a character doing their sign...
FOUR DIGIT NUMBERS
FOUR DIGIT NUMBERS• Tohandle four digit numbers, use the first two for the character and the second two as the action • 233...
FOUR DIGIT NUMBERS• Tohandle four digit numbers, use the first two for the character and the second two as the action  • 23...
LARGER NUMBERS
LARGER NUMBERS• “Chunk” up   bigger numbers into two and four digit groups
LARGER NUMBERS• “Chunk” up    bigger numbers into two and four digit groups• Build   wacky stories using those characters ...
A CAST OF CHARACTERS
A CAST OF CHARACTERShttp://ron.ludism.org/mnemonics_public.txt…40 = DO = Fred Flintstone, breaking rocks in the quarry (Ya...
A CAST OF CHARACTERShttp://ron.ludism.org/mnemonics_public.txt…40 = DO = Fred Flintstone, breaking rocks in the quarry (Ya...
BEANSTALKD (VIA STALKER)
BEANSTALKD
BEANSTALKD•A   simple but highly effective message queue
BEANSTALKD•A   simple but highly effective message queue• I’ve   used it many times, including to do moves in Go vs Go
BEANSTALKD•A   simple but highly effective message queue• I’ve   used it many times, including to do moves in Go vs Go• It...
CHARACTER LOOKUPS
CHARACTER LOOKUPSrequire "open-uri"def lookup_table return @lookup_table if defined? @lookup_table @lookup_table = { } open...
CHARACTER LOOKUPSrequire "open-uri"def lookup_table return @lookup_table if defined? @lookup_table @lookup_table = { } open...
CHARACTER LOOKUPSrequire "open-uri"def lookup_table return @lookup_table if defined? @lookup_table @lookup_table = { } open...
ENCODING CHUNKS
ENCODING CHUNKSdef encode(n, as = :both) if n.size == 4 and as == :both   "#{encode(n[0..1], :character)} #{encode(n[2..3]...
ENCODING CHUNKSdef encode(n, as = :both) if n.size == 4 and as == :both   "#{encode(n[0..1], :character)} #{encode(n[2..3]...
ENCODING CHUNKSdef encode(n, as = :both) if n.size == 4 and as == :both   "#{encode(n[0..1], :character)} #{encode(n[2..3]...
ENCODING NUMBERS
ENCODING NUMBERSjob "number.encode" do |args|  number = args["number"].to_s.delete("^0-9")  number = "0#{number}" if (numb...
ENCODING NUMBERSjob "number.encode" do |args|  number = args["number"].to_s.delete("^0-9")  number = "0#{number}" if (numb...
ENCODING NUMBERSjob "number.encode" do |args|  number = args["number"].to_s.delete("^0-9")  number = "0#{number}" if (numb...
ENCODING NUMBERSjob "number.encode" do |args|  number = args["number"].to_s.delete("^0-9")  number = "0#{number}" if (numb...
DISPLAYING ENCODINGS
DISPLAYING ENCODINGS  job "number.display" do |args|    Array(args["encodings"]).each do |encoded|     puts "#{args[number...
QUEUING NUMBERS
QUEUING NUMBERSabort "USAGE: #{$PROGRAM_NAME} NUMBER" if ARGV.empty?require "stalker"Stalker.enqueue("number.encode", numb...
RUNNING JOBS
RUNNING JOBS$ beanstalkd
RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536
RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 ...
RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 ...
RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 ...
RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 ...
RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 ...
RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 ...
RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 ...
FOR MORE INFORMATION      See Railscasts #243
FOR MORE INFORMATION      See Railscasts #243
BEANSTALKD PROS
BEANSTALKD PROS• Extremely   lightweight (especially if you drop the Rails stack)
BEANSTALKD PROS• Extremely   lightweight (especially if you drop the Rails stack)• Ridiculously   fast
BEANSTALKD PROS• Extremely   lightweight (especially if you drop the Rails stack)• Ridiculously   fast• Designed    to be ...
BEANSTALKD PROS• Extremely   lightweight (especially if you drop the Rails stack)• Ridiculously   fast• Designed    to be ...
BEANSTALKD PROS• Extremely    lightweight (especially if you drop the Rails stack)• Ridiculously   fast• Designed    to be...
BEANSTALKD CONS
BEANSTALKD CONS• Not ideal for durable queuing needs (though it does have a binlog)
BEANSTALKD CONS• Not ideal for durable queuing needs (though it does have a binlog)• Missing        niceties like logging ...
BEANSTALKD CONS• Not ideal for durable queuing needs (though it does have a binlog)• Missing        niceties like logging ...
0MQ(AKA ZEROMQ OR ZMQ)
0MQ
0MQ•A   high-level socket API over an asynchronous message queue • Request   and reply, publish subscribe, and pipeline mo...
0MQ•A   high-level socket API over an asynchronous message queue  • Request    and reply, publish subscribe, and pipeline ...
THE END OF THE PIPELINE
THE END OF THE PIPELINE     require "zmq"     zmq = ZMQ::Context.new     up = zmq.socket(ZMQ::UPSTREAM)     up.bind("tcp:/...
THE END OF THE PIPELINE     require "zmq"     zmq = ZMQ::Context.new     up = zmq.socket(ZMQ::UPSTREAM)     up.bind("tcp:/...
THE END OF THE PIPELINE     require "zmq"     zmq = ZMQ::Context.new     up = zmq.socket(ZMQ::UPSTREAM)     up.bind("tcp:/...
THE END OF THE PIPELINE     require "zmq"     zmq = ZMQ::Context.new     up = zmq.socket(ZMQ::UPSTREAM)     up.bind("tcp:/...
NUMBER LOOKUPS
NUMBER LOOKUPS# same lookup_table() as before…def lookup lookup_table.each do |number, character_and_action|   return numb...
NUMBER LOOKUPS# same lookup_table() as before…def lookup lookup_table.each do |number, character_and_action|   return numb...
A DECODER
A DECODERdef decode(message) message.to_s.strip.split(/.(?:s{2}|n|z)/).map { |chunk|   chunk.gsub!(/s+/, " ")   character ...
THE MIDDLE PIPE
THE MIDDLE PIPErequire "zmq"zmq = ZMQ::Context.newup = zmq.socket(ZMQ::UPSTREAM)down = zmq.socket(ZMQ::DOWNSTREAM)up.bind(...
THE MIDDLE PIPErequire "zmq"zmq = ZMQ::Context.newup = zmq.socket(ZMQ::UPSTREAM)down = zmq.socket(ZMQ::DOWNSTREAM)up.bind(...
THE MIDDLE PIPErequire "zmq"zmq = ZMQ::Context.newup = zmq.socket(ZMQ::UPSTREAM)down = zmq.socket(ZMQ::DOWNSTREAM)up.bind(...
THE MIDDLE PIPErequire "zmq"zmq = ZMQ::Context.newup = zmq.socket(ZMQ::UPSTREAM)down = zmq.socket(ZMQ::DOWNSTREAM)up.bind(...
THE PIPELINE OPENING
THE PIPELINE OPENING   require "zmq"   zmq = ZMQ::Context.new   down = zmq.socket(ZMQ::DOWNSTREAM)   down.connect("tcp://1...
A DATA PIPELINE IN ACTION
A DATA PIPELINE IN ACTION$ ruby queue_message.rbFred Flintstone breaking rocks in the quarry (Yabba dabba DOo!).Stuart Lit...
A DATA PIPELINE IN ACTION$ ruby queue_message.rbFred Flintstone breaking rocks in the quarry (Yabba dabba DOo!).Stuart Lit...
A DATA PIPELINE IN ACTION$ ruby queue_message.rbFred Flintstone breaking rocks in the quarry (Yabba dabba DOo!).Stuart Lit...
0MQ PROS
0MQ PROS• Asynchronous   messaging primitives for constructing networks
0MQ PROS• Asynchronous    messaging primitives for constructing networks• Extremely   versatile (can be made to fit most an...
0MQ PROS• Asynchronous    messaging primitives for constructing networks• Extremely   versatile (can be made to fit most an...
0MQ PROS• Asynchronous    messaging primitives for constructing networks• Extremely   versatile (can be made to fit most an...
0MQ CONS
0MQ CONS• Very   low-level (really just a platform to build on) • Missing   all the conveniences: logging, message formats...
0MQ CONS• Very   low-level (really just a platform to build on)  • Missing   all the conveniences: logging, message format...
THANKS!
In the Back of Your Mind
In the Back of Your Mind
In the Back of Your Mind
In the Back of Your Mind
Upcoming SlideShare
Loading in...5
×

In the Back of Your Mind

1,228

Published on

Some simple examples of how to pipeline data through beanstalkd or 0MQ, using Dominic memory encoding as an example.

Published in: Technology
5 Comments
2 Likes
Statistics
Notes
  • again: thank you for sharing & careing
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I agree with pretty much everything you've said. I've recently switched to using Speaker Deck, for putting my presentations online (for these and other reasons):

    http://speakerdeck.com/u/jeg2

    You can now download a PDF from there.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • my proposal would be to only provide pdf (easy to create from keynote).

    if you want to provide the whole presentation, you can publish a libreoffice/openoffice or microsoft office presentation. so there is no operation system barrier, but providing keynote only, you look out the other systems because opening keynote file on other systems is not as easy as the oposite way.

    or you can provide keynote or pdf here and upload the other one to a file sharing system or you can upload this presentation here twice.

    thank you for shareing your cool presentation, thank you for allowing to download the presentation, thank you for careing to answer, thank you for your work. go this one step further and be operationsystem independent.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hmm, I don't see a way for me to provide both. :(
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • pdf for non-mac user would be great
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
1,228
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
5
Likes
2
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • In the Back of Your Mind

    1. 1. IN THE BACK OF YOUR MIND beanstalkd and 0MQ through the Dominic System
    2. 2. THE DOMINIC SYSTEM
    3. 3. LEARN HOW TO IMPROVE YOUR MINDA book I am currently reading
    4. 4. MEMORY TECHNIQUES
    5. 5. MEMORY TECHNIQUES• The book includes various techniques for improving memory
    6. 6. MEMORY TECHNIQUES• The book includes various techniques for improving memory• Ithelps to translate abstract data (the brain is weak with) into vivid images (a strength of the brain)
    7. 7. MEMORY TECHNIQUES• The book includes various techniques for improving memory• Ithelps to translate abstract data (the brain is weak with) into vivid images (a strength of the brain)• The Dominic System is used to translate numbers into characters and actions • It can help you memorize large numbers
    8. 8. SINGLE DIGITS
    9. 9. SINGLE DIGITS• Just count through the alphabet • But replace F with S • And skip to N when you hit 9
    10. 10. SINGLE DIGITS 1 A 2 B• Just count through the 3 C alphabet 4 D 5 E • But replace F with S 6 S • And skip to N when you 7 G hit 9 8 H 9 N 0 O
    11. 11. DOUBLE DIGITS
    12. 12. DOUBLE DIGITS• Convert digit pairs into letters • 15 = AE, 23 = BC, 27 = BG, 39 = CN, 80 = HO, 80 = HA
    13. 13. DOUBLE DIGITS• Convert digit pairs into letters • 15 = AE, 23 = BC, 27 = BG, 39 = CN, 80 = HO, 80 = HA• Use the letters as initials for a character • AE = Albert Einstein, BG = Bill Gates, CN = Chuck Norris
    14. 14. DOUBLE DIGITS• Convert digit pairs into letters • 15 = AE, 23 = BC, 27 = BG, 39 = CN, 80 = HO, 80 = HA• Use the letters as initials for a character • AE = Albert Einstein, BG = Bill Gates, CN = Chuck Norris• Or use letters to indicate characters in other ways • BC = Jesus Christ, HO = Santa Claus, HA = Julia Sweeney
    15. 15. IMAGERY IS KEY
    16. 16. IMAGERY IS KEY• Each digit pair should create an image in your mind
    17. 17. IMAGERY IS KEY• Each digit pair should create an image in your mind• The image need to be a character doing their signature action • Chuck Norris performing a roundhouse kick • Santa Claus delivering presents • Julia Sweeney dressing asexually
    18. 18. FOUR DIGIT NUMBERS
    19. 19. FOUR DIGIT NUMBERS• Tohandle four digit numbers, use the first two for the character and the second two as the action • 2339 = BCCN = Jesus Christ performing a roundhouse kick • 8081 = HOHA = Santa Claus dressing asexually
    20. 20. FOUR DIGIT NUMBERS• Tohandle four digit numbers, use the first two for the character and the second two as the action • 2339 = BCCN = Jesus Christ performing a roundhouse kick • 8081 = HOHA = Santa Claus dressing asexually• You’re up to a 10,000 number memory at this point!
    21. 21. LARGER NUMBERS
    22. 22. LARGER NUMBERS• “Chunk” up bigger numbers into two and four digit groups
    23. 23. LARGER NUMBERS• “Chunk” up bigger numbers into two and four digit groups• Build wacky stories using those characters and actions • 27808039 = 2780 8039 = BGHO HOCN = Bill Gates started delivering presents, but this angered Santa Claus who gave him a roundhouse kick.
    24. 24. A CAST OF CHARACTERS
    25. 25. A CAST OF CHARACTERShttp://ron.ludism.org/mnemonics_public.txt…40 = DO = Fred Flintstone, breaking rocks in the quarry (Yabba dabba DOo!)41 = DA = Douglas Adams, putting a Babelfish into his ear (_The Hitch Hikers Guide to the Galaxy_)42 = DB = PERSONAL43 = DC = George Washington, chopping down cherry tree (Washington DC)44 = DD = Danny Dunn (fictional character), descending in a bathysphere45 = DE = PERSONAL46 = DS = Ivan Stang (a.k.a. Doug Smith), ranting on a soapbox47 = DG = PERSONAL48 = DH = Douglas Hofstadter, recursively holding a picture of himself49 = DN = Dan (_Roseanne_ TV show), wearing sunglasses to switch on Christmas lights…
    26. 26. A CAST OF CHARACTERShttp://ron.ludism.org/mnemonics_public.txt…40 = DO = Fred Flintstone, breaking rocks in the quarry (Yabba dabba DOo!)41 = DA = Douglas Adams, putting a Babelfish into his ear (_The Hitch Hikers Guide to the Galaxy_)42 = DB = PERSONAL43 = DC = George Washington, chopping down cherry tree (Washington DC)44 = DD = Danny Dunn (fictional character), descending in a bathysphere45 = DE = PERSONAL46 = DS = Ivan Stang (a.k.a. Doug Smith), ranting on a soapbox47 = DG = PERSONAL48 = DH = Douglas Hofstadter, recursively holding a picture of himself49 = DN = Dan (_Roseanne_ TV show), wearing sunglasses to switch on Christmas lights…
    27. 27. BEANSTALKD (VIA STALKER)
    28. 28. BEANSTALKD
    29. 29. BEANSTALKD•A simple but highly effective message queue
    30. 30. BEANSTALKD•A simple but highly effective message queue• I’ve used it many times, including to do moves in Go vs Go
    31. 31. BEANSTALKD•A simple but highly effective message queue• I’ve used it many times, including to do moves in Go vs Go• It’s definitely one of my favorite tools for background jobs
    32. 32. CHARACTER LOOKUPS
    33. 33. CHARACTER LOOKUPSrequire "open-uri"def lookup_table return @lookup_table if defined? @lookup_table @lookup_table = { } open("http://ron.ludism.org/mnemonics_public.txt") do |page| page.each do |line| if line =~ /A(d{2})s*=s*[A-Z]{2}s*=s*(.+)/ @lookup_table[$1] = $2 elsif line =~ /As+(.+)/ @lookup_table.values.last << " #{$1}" end end end @lookup_tableend
    34. 34. CHARACTER LOOKUPSrequire "open-uri"def lookup_table return @lookup_table if defined? @lookup_table @lookup_table = { } open("http://ron.ludism.org/mnemonics_public.txt") do |page| page.each do |line| if line =~ /A(d{2})s*=s*[A-Z]{2}s*=s*(.+)/ @lookup_table[$1] = $2 elsif line =~ /As+(.+)/ @lookup_table.values.last << " #{$1}" end end end @lookup_tableend
    35. 35. CHARACTER LOOKUPSrequire "open-uri"def lookup_table return @lookup_table if defined? @lookup_table @lookup_table = { } open("http://ron.ludism.org/mnemonics_public.txt") do |page| page.each do |line| if line =~ /A(d{2})s*=s*[A-Z]{2}s*=s*(.+)/ @lookup_table[$1] = $2 elsif line =~ /As+(.+)/ @lookup_table.values.last << " #{$1}" end end end @lookup_tableend
    36. 36. ENCODING CHUNKS
    37. 37. ENCODING CHUNKSdef encode(n, as = :both) if n.size == 4 and as == :both "#{encode(n[0..1], :character)} #{encode(n[2..3], :action)}." elsif as == :both "#{encode(n, :character)} #{encode(n, :action)}." elsif as == :character lookup_table[n][/As*([^,]+(?:([^)]+))?)/, 1] elsif as == :action lookup_table[n][/As*[^,]+(?:([^)]+))?,s*(.+?)s*z/, 1] endend
    38. 38. ENCODING CHUNKSdef encode(n, as = :both) if n.size == 4 and as == :both "#{encode(n[0..1], :character)} #{encode(n[2..3], :action)}." elsif as == :both "#{encode(n, :character)} #{encode(n, :action)}." elsif as == :character lookup_table[n][/As*([^,]+(?:([^)]+))?)/, 1] elsif as == :action lookup_table[n][/As*[^,]+(?:([^)]+))?,s*(.+?)s*z/, 1] endend
    39. 39. ENCODING CHUNKSdef encode(n, as = :both) if n.size == 4 and as == :both "#{encode(n[0..1], :character)} #{encode(n[2..3], :action)}." elsif as == :both "#{encode(n, :character)} #{encode(n, :action)}." elsif as == :character lookup_table[n][/As*([^,]+(?:([^)]+))?)/, 1] elsif as == :action lookup_table[n][/As*[^,]+(?:([^)]+))?,s*(.+?)s*z/, 1] endend
    40. 40. ENCODING NUMBERS
    41. 41. ENCODING NUMBERSjob "number.encode" do |args| number = args["number"].to_s.delete("^0-9") number = "0#{number}" if (number.size % 2).nonzero? chunks = [4] * (number.size / 4) + [2] * (number.size % 4 / 2) encodings = [ ] chunks.permutation.to_a.uniq.each do |grouping| encodings << number.match(/A#{grouping.map { |g| "(.{#{g}})" }.join}z/) .captures .map { |n| encode(n) } .join(" ") .tr("_", ") end Stalker.enqueue( "number.display", number: args["number"], encodings: encodings )end
    42. 42. ENCODING NUMBERSjob "number.encode" do |args| number = args["number"].to_s.delete("^0-9") number = "0#{number}" if (number.size % 2).nonzero? chunks = [4] * (number.size / 4) + [2] * (number.size % 4 / 2) encodings = [ ] chunks.permutation.to_a.uniq.each do |grouping| encodings << number.match(/A#{grouping.map { |g| "(.{#{g}})" }.join}z/) .captures .map { |n| encode(n) } .join(" ") .tr("_", ") end Stalker.enqueue( "number.display", number: args["number"], encodings: encodings )end
    43. 43. ENCODING NUMBERSjob "number.encode" do |args| number = args["number"].to_s.delete("^0-9") number = "0#{number}" if (number.size % 2).nonzero? chunks = [4] * (number.size / 4) + [2] * (number.size % 4 / 2) encodings = [ ] chunks.permutation.to_a.uniq.each do |grouping| encodings << number.match(/A#{grouping.map { |g| "(.{#{g}})" }.join}z/) .captures .map { |n| encode(n) } .join(" ") .tr("_", ") end Stalker.enqueue( "number.display", number: args["number"], encodings: encodings )end
    44. 44. ENCODING NUMBERSjob "number.encode" do |args| number = args["number"].to_s.delete("^0-9") number = "0#{number}" if (number.size % 2).nonzero? chunks = [4] * (number.size / 4) + [2] * (number.size % 4 / 2) encodings = [ ] chunks.permutation.to_a.uniq.each do |grouping| encodings << number.match(/A#{grouping.map { |g| "(.{#{g}})" }.join}z/) .captures .map { |n| encode(n) } .join(" ") .tr("_", ") end Stalker.enqueue( "number.display", number: args["number"], encodings: encodings )end
    45. 45. DISPLAYING ENCODINGS
    46. 46. DISPLAYING ENCODINGS job "number.display" do |args| Array(args["encodings"]).each do |encoded| puts "#{args[number]}:" puts encoded puts end end
    47. 47. QUEUING NUMBERS
    48. 48. QUEUING NUMBERSabort "USAGE: #{$PROGRAM_NAME} NUMBER" if ARGV.empty?require "stalker"Stalker.enqueue("number.encode", number: ARGV.shift)
    49. 49. RUNNING JOBS
    50. 50. RUNNING JOBS$ beanstalkd
    51. 51. RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536
    52. 52. RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 13:07:24 -0600] Working 1 jobs: [ number.encode ][2010-12-08 13:11:18 -0600] -> number.encode (number=405-285-0536)[2010-12-08 13:11:19 -0600] -> number.encode finished in 510ms
    53. 53. RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 13:07:24 -0600] Working 1 jobs: [ number.encode ][2010-12-08 13:11:18 -0600] -> number.encode (number=405-285-0536)[2010-12-08 13:11:19 -0600] -> number.encode finished in 510ms
    54. 54. RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 13:07:24 -0600] Working 1 jobs: [ number.encode ][2010-12-08 13:11:18 -0600] -> number.encode (number=405-285-0536)[2010-12-08 13:11:19 -0600] -> number.encode finished in 510ms
    55. 55. RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 13:07:24 -0600] Working 1 jobs: [ number.encode ][2010-12-08 13:11:18 -0600] -> number.encode (number=405-285-0536)[2010-12-08 13:11:19 -0600] -> number.encode finished in 510ms
    56. 56. RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 13:07:24 -0600] Working 1 jobs: [ number.encode ][2010-12-08 13:11:18 -0600] -> number.encode (number=405-285-0536)[2010-12-08 13:11:19 -0600] -> number.encode finished in 510ms
    57. 57. RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 13:07:24 -0600] Working 1 jobs: [ number.encode ][2010-12-08 13:11:18 -0600] -> number.encode (number=405-285-0536)[2010-12-08 13:11:19 -0600] -> number.encode finished in 510ms
    58. 58. RUNNING JOBS$ beanstalkd$ ruby encode_number.rb 405-285-0536$ rvm 1.9.2$ RUBYLIB=. stalk jobs.rb number.encode[2010-12-08 13:07:24 -0600] Working 1 jobs: [ number.encode ][2010-12-08 13:11:18 -0600] -> number.encode (number=405-285-0536)[2010-12-08 13:11:19 -0600] -> number.encode finished in 510ms$ RUBYLIB=. stalk jobs.rb number.display…405-285-0536:Fred Flintstone breaking rocks in the quarry (Yabba dabba DOo!).Stuart Little (tiny fictional mouse wearing street clothes) releasinghelium balloons (Newtonmas, HE HE HE!). J.R.R. Tolkien stepping intothe Wardrobe.…
    59. 59. FOR MORE INFORMATION See Railscasts #243
    60. 60. FOR MORE INFORMATION See Railscasts #243
    61. 61. BEANSTALKD PROS
    62. 62. BEANSTALKD PROS• Extremely lightweight (especially if you drop the Rails stack)
    63. 63. BEANSTALKD PROS• Extremely lightweight (especially if you drop the Rails stack)• Ridiculously fast
    64. 64. BEANSTALKD PROS• Extremely lightweight (especially if you drop the Rails stack)• Ridiculously fast• Designed to be a queue
    65. 65. BEANSTALKD PROS• Extremely lightweight (especially if you drop the Rails stack)• Ridiculously fast• Designed to be a queue• “Tubes” make it trivial to divide work or pipeline jobs
    66. 66. BEANSTALKD PROS• Extremely lightweight (especially if you drop the Rails stack)• Ridiculously fast• Designed to be a queue• “Tubes” make it trivial to divide work or pipeline jobs• Really easy to spread across ruby versions and environments
    67. 67. BEANSTALKD CONS
    68. 68. BEANSTALKD CONS• Not ideal for durable queuing needs (though it does have a binlog)
    69. 69. BEANSTALKD CONS• Not ideal for durable queuing needs (though it does have a binlog)• Missing niceties like logging (though Stalker adds logging above beanstalkd)
    70. 70. BEANSTALKD CONS• Not ideal for durable queuing needs (though it does have a binlog)• Missing niceties like logging (though Stalker adds logging above beanstalkd)• Needs job introspection
    71. 71. 0MQ(AKA ZEROMQ OR ZMQ)
    72. 72. 0MQ
    73. 73. 0MQ•A high-level socket API over an asynchronous message queue • Request and reply, publish subscribe, and pipeline models
    74. 74. 0MQ•A high-level socket API over an asynchronous message queue • Request and reply, publish subscribe, and pipeline models• This was my first time using it (I was curious about it) •I don’t think it’s well suited to this task
    75. 75. THE END OF THE PIPELINE
    76. 76. THE END OF THE PIPELINE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) up.bind("tcp://127.0.0.1:5000") while message = up.recv puts message end
    77. 77. THE END OF THE PIPELINE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) up.bind("tcp://127.0.0.1:5000") while message = up.recv puts message end
    78. 78. THE END OF THE PIPELINE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) up.bind("tcp://127.0.0.1:5000") while message = up.recv puts message end
    79. 79. THE END OF THE PIPELINE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) up.bind("tcp://127.0.0.1:5000") while message = up.recv puts message end
    80. 80. NUMBER LOOKUPS
    81. 81. NUMBER LOOKUPS# same lookup_table() as before…def lookup lookup_table.each do |number, character_and_action| return number if yield character_and_action end nilenddef character_number(msg) lookup { |ca| msg.start_with? ca[/As*([^,]+(?:([^)]+))?)/, 1] }enddef action_number(msg) lookup { |ca| msg.end_with? ca[/As*[^,]+(?:([^)]+))?,s*(.+?)s*z/, 1] }end
    82. 82. NUMBER LOOKUPS# same lookup_table() as before…def lookup lookup_table.each do |number, character_and_action| return number if yield character_and_action end nilenddef character_number(msg) lookup { |ca| msg.start_with? ca[/As*([^,]+(?:([^)]+))?)/, 1] }enddef action_number(msg) lookup { |ca| msg.end_with? ca[/As*[^,]+(?:([^)]+))?,s*(.+?)s*z/, 1] }end
    83. 83. A DECODER
    84. 84. A DECODERdef decode(message) message.to_s.strip.split(/.(?:s{2}|n|z)/).map { |chunk| chunk.gsub!(/s+/, " ") character = character_number(chunk) action = action_number(chunk) character == action ? character : "#{character}#{action}" }.joinend
    85. 85. THE MIDDLE PIPE
    86. 86. THE MIDDLE PIPErequire "zmq"zmq = ZMQ::Context.newup = zmq.socket(ZMQ::UPSTREAM)down = zmq.socket(ZMQ::DOWNSTREAM)up.bind("tcp://127.0.0.1:5001")down.connect("tcp://127.0.0.1:5000")while message = up.recv down.send(decode(message))end
    87. 87. THE MIDDLE PIPErequire "zmq"zmq = ZMQ::Context.newup = zmq.socket(ZMQ::UPSTREAM)down = zmq.socket(ZMQ::DOWNSTREAM)up.bind("tcp://127.0.0.1:5001")down.connect("tcp://127.0.0.1:5000")while message = up.recv down.send(decode(message))end
    88. 88. THE MIDDLE PIPErequire "zmq"zmq = ZMQ::Context.newup = zmq.socket(ZMQ::UPSTREAM)down = zmq.socket(ZMQ::DOWNSTREAM)up.bind("tcp://127.0.0.1:5001")down.connect("tcp://127.0.0.1:5000")while message = up.recv down.send(decode(message))end
    89. 89. THE MIDDLE PIPErequire "zmq"zmq = ZMQ::Context.newup = zmq.socket(ZMQ::UPSTREAM)down = zmq.socket(ZMQ::DOWNSTREAM)up.bind("tcp://127.0.0.1:5001")down.connect("tcp://127.0.0.1:5000")while message = up.recv down.send(decode(message))end
    90. 90. THE PIPELINE OPENING
    91. 91. THE PIPELINE OPENING require "zmq" zmq = ZMQ::Context.new down = zmq.socket(ZMQ::DOWNSTREAM) down.connect("tcp://127.0.0.1:5001") down.send(ARGF.read)
    92. 92. A DATA PIPELINE IN ACTION
    93. 93. A DATA PIPELINE IN ACTION$ ruby queue_message.rbFred Flintstone breaking rocks in the quarry (Yabba dabba DOo!).Stuart Little (tiny fictional mouse wearing street clothes) releasinghelium balloons (Newtonmas, HE HE HE!). J.R.R. Tolkien stepping intothe Wardrobe.^d
    94. 94. A DATA PIPELINE IN ACTION$ ruby queue_message.rbFred Flintstone breaking rocks in the quarry (Yabba dabba DOo!).Stuart Little (tiny fictional mouse wearing street clothes) releasinghelium balloons (Newtonmas, HE HE HE!). J.R.R. Tolkien stepping intothe Wardrobe.^d $ ruby decoder.rb
    95. 95. A DATA PIPELINE IN ACTION$ ruby queue_message.rbFred Flintstone breaking rocks in the quarry (Yabba dabba DOo!).Stuart Little (tiny fictional mouse wearing street clothes) releasinghelium balloons (Newtonmas, HE HE HE!). J.R.R. Tolkien stepping intothe Wardrobe.^d $ ruby decoder.rb $ ruby decoder_display.rb 4052850536
    96. 96. 0MQ PROS
    97. 97. 0MQ PROS• Asynchronous messaging primitives for constructing networks
    98. 98. 0MQ PROS• Asynchronous messaging primitives for constructing networks• Extremely versatile (can be made to fit most any model)
    99. 99. 0MQ PROS• Asynchronous messaging primitives for constructing networks• Extremely versatile (can be made to fit most any model)• Supports many to many connections
    100. 100. 0MQ PROS• Asynchronous messaging primitives for constructing networks• Extremely versatile (can be made to fit most any model)• Supports many to many connections• Automatically load balanced
    101. 101. 0MQ CONS
    102. 102. 0MQ CONS• Very low-level (really just a platform to build on) • Missing all the conveniences: logging, message formats, etc.
    103. 103. 0MQ CONS• Very low-level (really just a platform to build on) • Missing all the conveniences: logging, message formats, etc.• Best for asynchronous networks you design and build
    104. 104. THANKS!
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×