Gotcha! Ruby things that will come back to bite you.
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,329
On Slideshare
1,159
From Embeds
170
Number of Embeds
7

Actions

Shares
Downloads
9
Comments
0
Likes
1

Embeds 170

http://blog.burtcorp.com 94
http://lanyrd.com 70
http://reader128k.net 2
http://www.hanrss.com 1
http://coderwall.com 1
http://feeds.feedburner.com 1
https://abs.twimg.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • Quick show of hands\nWho here runs a ruby system in production?\nWhat about jruby?\nWho runs an advanced jruby system, as in using java modules, threading, java specific gems, that sort of thing?\n
  • I’m David, and i’m the lead platform developer at Burt\nThat means i’m scrum master and i run the day to day development of the platform team\n\nBurt is an ad analytics startup. Simply put we track ads on your website, crunch a whole lot of data about it and present it in nice reports.\nThe platform team that i’m in runs the whole crunching pipeline, and we process about 15k requests per second at peak times, in real time. \nWe use various versions of JRuby 1.6 in Ruby 1.9 mode, along with tools like RabbitMQ, MongoDB, Redis and Cassandra.\n\n
  • I’m David, and i’m the lead platform developer at Burt\nThat means i’m scrum master and i run the day to day development of the platform team\n\nBurt is an ad analytics startup. Simply put we track ads on your website, crunch a whole lot of data about it and present it in nice reports.\nThe platform team that i’m in runs the whole crunching pipeline, and we process about 15k requests per second at peak times, in real time. \nWe use various versions of JRuby 1.6 in Ruby 1.9 mode, along with tools like RabbitMQ, MongoDB, Redis and Cassandra.\n\n
  • I’m David, and i’m the lead platform developer at Burt\nThat means i’m scrum master and i run the day to day development of the platform team\n\nBurt is an ad analytics startup. Simply put we track ads on your website, crunch a whole lot of data about it and present it in nice reports.\nThe platform team that i’m in runs the whole crunching pipeline, and we process about 15k requests per second at peak times, in real time. \nWe use various versions of JRuby 1.6 in Ruby 1.9 mode, along with tools like RabbitMQ, MongoDB, Redis and Cassandra.\n\n
  • I’m David, and i’m the lead platform developer at Burt\nThat means i’m scrum master and i run the day to day development of the platform team\n\nBurt is an ad analytics startup. Simply put we track ads on your website, crunch a whole lot of data about it and present it in nice reports.\nThe platform team that i’m in runs the whole crunching pipeline, and we process about 15k requests per second at peak times, in real time. \nWe use various versions of JRuby 1.6 in Ruby 1.9 mode, along with tools like RabbitMQ, MongoDB, Redis and Cassandra.\n\n
  • I’m David, and i’m the lead platform developer at Burt\nThat means i’m scrum master and i run the day to day development of the platform team\n\nBurt is an ad analytics startup. Simply put we track ads on your website, crunch a whole lot of data about it and present it in nice reports.\nThe platform team that i’m in runs the whole crunching pipeline, and we process about 15k requests per second at peak times, in real time. \nWe use various versions of JRuby 1.6 in Ruby 1.9 mode, along with tools like RabbitMQ, MongoDB, Redis and Cassandra.\n\n
  • What i’m going to talk about are some of the things we’ve learned when building and optimizing our system. \nOur use case is very specific, and not everything i’m going to talk about applies to everyone.\nIt’s a very different thing writing a long running application as opposed to a web application where every session is very short lived. \n\n
  • What i’m going to talk about are some of the things we’ve learned when building and optimizing our system. \nOur use case is very specific, and not everything i’m going to talk about applies to everyone.\nIt’s a very different thing writing a long running application as opposed to a web application where every session is very short lived. \n\n
  • What i’m going to talk about are some of the things we’ve learned when building and optimizing our system. \nOur use case is very specific, and not everything i’m going to talk about applies to everyone.\nIt’s a very different thing writing a long running application as opposed to a web application where every session is very short lived. \n\n
  • What i’m going to talk about are some of the things we’ve learned when building and optimizing our system. \nOur use case is very specific, and not everything i’m going to talk about applies to everyone.\nIt’s a very different thing writing a long running application as opposed to a web application where every session is very short lived. \n\n
  • What i’m going to talk about are some of the things we’ve learned when building and optimizing our system. \nOur use case is very specific, and not everything i’m going to talk about applies to everyone.\nIt’s a very different thing writing a long running application as opposed to a web application where every session is very short lived. \n\n
  • First thing i want to talk about are strings.\nIt’s something we all use, all the time\nBut there’s one big pitfall\n\n
  • Writing a string literal in Ruby creates a new string.\nWhich means that this (“”), is the same as this (String.new)\nMost of the time, this isn’t a problem, but if you do something like this that could be trouble\n\nThe problem here is that when looking at this code, you get a feeling that string_key is the same string all the time. But it’s not, there’s a new string created on every iteration of this loop. This is because strings in ruby are mutable, as compared to languages like Java, C# or Python.\n
  • Writing a string literal in Ruby creates a new string.\nWhich means that this (“”), is the same as this (String.new)\nMost of the time, this isn’t a problem, but if you do something like this that could be trouble\n\nThe problem here is that when looking at this code, you get a feeling that string_key is the same string all the time. But it’s not, there’s a new string created on every iteration of this loop. This is because strings in ruby are mutable, as compared to languages like Java, C# or Python.\n
  • Writing a string literal in Ruby creates a new string.\nWhich means that this (“”), is the same as this (String.new)\nMost of the time, this isn’t a problem, but if you do something like this that could be trouble\n\nThe problem here is that when looking at this code, you get a feeling that string_key is the same string all the time. But it’s not, there’s a new string created on every iteration of this loop. This is because strings in ruby are mutable, as compared to languages like Java, C# or Python.\n
  • What you want to do instead is putting that string in a constant, and then use the constant when working with the hash. Or even better, use a symbol or an int.\n\nHere are some benchmarks for doing this a million times with different keys.\nAs you can see, the string version is slowest, almost twice as slow as using ints as keys\n\nObviously, some of this time goes to calculating the hash codes, which is why int is so much faster than the others.\n\n
  • Now, the really interesting difference becomes apparent if we turn on verbose garbage collection\n\nThis is the printout for a minute run of a similar loop using a string key\nAs you can see, the GC has to work quite a lot.\nIn comparison, there is NO GC activity at all when using a constant or a symbol\n\n
  • This is what the memory use looks like.\nNow, in a simple loop the effect of this is quite small. The GC doesn’t really have any problems dealing with it. But once you put this in a complex application along with several other threads and other stuff, and then let it run for a couple of days, this can cause a significant slowdown.\n
  • As a comparison, the upper graph shows what the memory use looks like with a constant. Do note that the upper limit for that instance is 25 mb, compared to the string example that fluctuates between 25 and 275 mb.\n\nBtw, these graphs are from VisualVM, which is an excellent tool for looking into performance and memory issues in java, and it works great for Jruby too\n
  • \n
  • The next thing is something we discovered recently.\nWe had a spot where we took a bunch of different variables, stuck them in an array and joined them together into a coordinate like this\n\nWhen we profiled the code we saw a LOT of calls to method_missing which didn’t make any sense, and it was a real hot spot, like 15-20% of the processing time\n\nTurns out, Array.join uses to_str, not to_s. Which when you think about it makes a lot of sense.\nto_str is an implicit cast, or a more strict conversion. You should only implement it if your object can naturally be used every place a string could be used.\nSo, to_str is not implemented on symbols, fixnum or nilclass.\n\nBut, ruby is clever! So when to_str doesn’t exist, it goes down to method missing and eventually finds it way to to_s. But as you can see, thats quite a call stack being generated. So there’s a lot of overhead.\n
  • The next thing is something we discovered recently.\nWe had a spot where we took a bunch of different variables, stuck them in an array and joined them together into a coordinate like this\n\nWhen we profiled the code we saw a LOT of calls to method_missing which didn’t make any sense, and it was a real hot spot, like 15-20% of the processing time\n\nTurns out, Array.join uses to_str, not to_s. Which when you think about it makes a lot of sense.\nto_str is an implicit cast, or a more strict conversion. You should only implement it if your object can naturally be used every place a string could be used.\nSo, to_str is not implemented on symbols, fixnum or nilclass.\n\nBut, ruby is clever! So when to_str doesn’t exist, it goes down to method missing and eventually finds it way to to_s. But as you can see, thats quite a call stack being generated. So there’s a lot of overhead.\n
  • This is a benchmark of joining an array of 1000 entries ten thousand times.\nAs you can see there’s quite a performance hit\n\nOne thing you can do to combat this is to map to_str to to_s. As you can see the overhead is lower, but to_s is still slow.\nBUT, this is not a good idea. Why? \n\n
  • This is a benchmark of joining an array of 1000 entries ten thousand times.\nAs you can see there’s quite a performance hit\n\nOne thing you can do to combat this is to map to_str to to_s. As you can see the overhead is lower, but to_s is still slow.\nBUT, this is not a good idea. Why? \n\n
  • Struct is the reason. If you’re not familiar with Struct, it’s a way of bundling attributes together without creating a full class.\nStruct has an optional string argument that names the class. And this is detected with duck-typing by checking to_str.\n\nSo if you send in an object responding to to_str as the first argument, Struct will demand that it behaves like a constant and starts with a capital letter. So if you created an anonymous structure with a symbol, and then added to_str to symbol, you could guess what happens.\n\nAnd the thing is, there’s a lot of gems using anonymous structs. Like webbrick. So if you have this to_str hack, and then decide to require webbrick, everything will crash.\n\n\n\n
  • So what can we do instead? A simple solution is to convert every item in the array to a string explicitly before joining.\nIt’s not quite as fast as implementing alias_method, but it’s non-invasive and improves performance.\n
  • So what can we do instead? A simple solution is to convert every item in the array to a string explicitly before joining.\nIt’s not quite as fast as implementing alias_method, but it’s non-invasive and improves performance.\n
  • \n
  • My next example comes from troubles we had with an internal FIFO queue.\nWhat we discovered was that when the internal buffer grew, we saw diminishing performance up to a point where it was impossible to catch up.\n\nWhat happens is that shift and unshift on an array causes the entire array to be moved in memory, and that’s quite costly if there’s a lot of entries.\n\nThere are a couple of ways to do this better. You could use a fixed size array with a sweeping index.\nOr you could go to Java and used a LinkedList. If you’re using threads, this is the way you want to go in the first place. Go straight for a LinkedBlockingQueue.\n
  • My next example comes from troubles we had with an internal FIFO queue.\nWhat we discovered was that when the internal buffer grew, we saw diminishing performance up to a point where it was impossible to catch up.\n\nWhat happens is that shift and unshift on an array causes the entire array to be moved in memory, and that’s quite costly if there’s a lot of entries.\n\nThere are a couple of ways to do this better. You could use a fixed size array with a sweeping index.\nOr you could go to Java and used a LinkedList. If you’re using threads, this is the way you want to go in the first place. Go straight for a LinkedBlockingQueue.\n
  • My next example comes from troubles we had with an internal FIFO queue.\nWhat we discovered was that when the internal buffer grew, we saw diminishing performance up to a point where it was impossible to catch up.\n\nWhat happens is that shift and unshift on an array causes the entire array to be moved in memory, and that’s quite costly if there’s a lot of entries.\n\nThere are a couple of ways to do this better. You could use a fixed size array with a sweeping index.\nOr you could go to Java and used a LinkedList. If you’re using threads, this is the way you want to go in the first place. Go straight for a LinkedBlockingQueue.\n
  • My next example comes from troubles we had with an internal FIFO queue.\nWhat we discovered was that when the internal buffer grew, we saw diminishing performance up to a point where it was impossible to catch up.\n\nWhat happens is that shift and unshift on an array causes the entire array to be moved in memory, and that’s quite costly if there’s a lot of entries.\n\nThere are a couple of ways to do this better. You could use a fixed size array with a sweeping index.\nOr you could go to Java and used a LinkedList. If you’re using threads, this is the way you want to go in the first place. Go straight for a LinkedBlockingQueue.\n
  • My next example comes from troubles we had with an internal FIFO queue.\nWhat we discovered was that when the internal buffer grew, we saw diminishing performance up to a point where it was impossible to catch up.\n\nWhat happens is that shift and unshift on an array causes the entire array to be moved in memory, and that’s quite costly if there’s a lot of entries.\n\nThere are a couple of ways to do this better. You could use a fixed size array with a sweeping index.\nOr you could go to Java and used a LinkedList. If you’re using threads, this is the way you want to go in the first place. Go straight for a LinkedBlockingQueue.\n
  • My next example comes from troubles we had with an internal FIFO queue.\nWhat we discovered was that when the internal buffer grew, we saw diminishing performance up to a point where it was impossible to catch up.\n\nWhat happens is that shift and unshift on an array causes the entire array to be moved in memory, and that’s quite costly if there’s a lot of entries.\n\nThere are a couple of ways to do this better. You could use a fixed size array with a sweeping index.\nOr you could go to Java and used a LinkedList. If you’re using threads, this is the way you want to go in the first place. Go straight for a LinkedBlockingQueue.\n
  • This is an example of a sweeping index array queue. You have a fixed size array and two indexes. And data between those indexes.\n
  • Then you pop a few items from the queue, and the start index is increased.\n
  • And if you push items to the array and you run over the edge, you start from the beginning again.\n
  • As you can see, for small queues the difference is minimal\nBut here’s what happens when we grow the queue\n\n
  • As you can see, for small queues the difference is minimal\nBut here’s what happens when we grow the queue\n\n
  • Now i’m gonna stop talking about you should avoid, and show you some good things to look at instead\n
  • There’s really no reason not to use a thread pool.\nNative threads have move overhead and are harder to control. Executor\n
  • There’s really no reason not to use a thread pool.\nNative threads have move overhead and are harder to control.\n\nYou should really go listen to Theo later, cause he will more about this.\n
  • If you find yourself using mutex, you’re probably doing it wrong\nSure, there are times where you REALLY need to lock, but most of the time you just need a good concurrent data structure\n\nJavas concurrency library is really good and probably has exactly what you need,\nlike atomic numerics, concurrent maps and lists\n\nAnd lastly Semaphore is a nicer and more feature complete locking mechanism if you ACTUALLY need locking. But again, think a couple of times before going for that.\n
  • If you find yourself using mutex, you’re probably doing it wrong\nSure, there are times where you REALLY need to lock, but most of the time you just need a good concurrent data structure\n\nJavas concurrency library is really good and probably has exactly what you need,\nlike atomic numerics, concurrent maps and lists\n\nAnd lastly Semaphore is a nicer and more feature complete locking mechanism if you ACTUALLY need locking. But again, think a couple of times before going for that.\n
  • If you find yourself using mutex, you’re probably doing it wrong\nSure, there are times where you REALLY need to lock, but most of the time you just need a good concurrent data structure\n\nJavas concurrency library is really good and probably has exactly what you need,\nlike atomic numerics, concurrent maps and lists\n\nAnd lastly Semaphore is a nicer and more feature complete locking mechanism if you ACTUALLY need locking. But again, think a couple of times before going for that.\n
  • If you find yourself using mutex, you’re probably doing it wrong\nSure, there are times where you REALLY need to lock, but most of the time you just need a good concurrent data structure\n\nJavas concurrency library is really good and probably has exactly what you need,\nlike atomic numerics, concurrent maps and lists\n\nAnd lastly Semaphore is a nicer and more feature complete locking mechanism if you ACTUALLY need locking. But again, think a couple of times before going for that.\n
  • If you find yourself using mutex, you’re probably doing it wrong\nSure, there are times where you REALLY need to lock, but most of the time you just need a good concurrent data structure\n\nJavas concurrency library is really good and probably has exactly what you need,\nlike atomic numerics, concurrent maps and lists\n\nAnd lastly Semaphore is a nicer and more feature complete locking mechanism if you ACTUALLY need locking. But again, think a couple of times before going for that.\n
  • \n
  • Shelling out is quite a lot slower in JRuby, so use in moderation. Most things don’t need shelling. For example this you can do with FileUtils.mkdir_p\nAlso look out for things like rake and capistrano shelling out\n\nPassing RubyStrings into java can be tricky, since they will be converted to java strings and you can get problems with string encoding. This is particulary important if the string isn’t really a string but a serialised object.\n\nFor example, Hot bunnies, which uses the rabbitMQ java library and wrapping it in a gem. When publishing data to rmq you need to serialize your data, send that string to java, which sends it across the network, then back into java and finally into ruby. \n
  • Shelling out is quite a lot slower in JRuby, so use in moderation. Most things don’t need shelling. For example this you can do with FileUtils.mkdir_p\nAlso look out for things like rake and capistrano shelling out\n\nPassing RubyStrings into java can be tricky, since they will be converted to java strings and you can get problems with string encoding. This is particulary important if the string isn’t really a string but a serialised object.\n\nFor example, Hot bunnies, which uses the rabbitMQ java library and wrapping it in a gem. When publishing data to rmq you need to serialize your data, send that string to java, which sends it across the network, then back into java and finally into ruby. \n
  • Shelling out is quite a lot slower in JRuby, so use in moderation. Most things don’t need shelling. For example this you can do with FileUtils.mkdir_p\nAlso look out for things like rake and capistrano shelling out\n\nPassing RubyStrings into java can be tricky, since they will be converted to java strings and you can get problems with string encoding. This is particulary important if the string isn’t really a string but a serialised object.\n\nFor example, Hot bunnies, which uses the rabbitMQ java library and wrapping it in a gem. When publishing data to rmq you need to serialize your data, send that string to java, which sends it across the network, then back into java and finally into ruby. \n
  • Shelling out is quite a lot slower in JRuby, so use in moderation. Most things don’t need shelling. For example this you can do with FileUtils.mkdir_p\nAlso look out for things like rake and capistrano shelling out\n\nPassing RubyStrings into java can be tricky, since they will be converted to java strings and you can get problems with string encoding. This is particulary important if the string isn’t really a string but a serialised object.\n\nFor example, Hot bunnies, which uses the rabbitMQ java library and wrapping it in a gem. When publishing data to rmq you need to serialize your data, send that string to java, which sends it across the network, then back into java and finally into ruby. \n
  • Shelling out is quite a lot slower in JRuby, so use in moderation. Most things don’t need shelling. For example this you can do with FileUtils.mkdir_p\nAlso look out for things like rake and capistrano shelling out\n\nPassing RubyStrings into java can be tricky, since they will be converted to java strings and you can get problems with string encoding. This is particulary important if the string isn’t really a string but a serialised object.\n\nFor example, Hot bunnies, which uses the rabbitMQ java library and wrapping it in a gem. When publishing data to rmq you need to serialize your data, send that string to java, which sends it across the network, then back into java and finally into ruby. \n
  • Shelling out is quite a lot slower in JRuby, so use in moderation. Most things don’t need shelling. For example this you can do with FileUtils.mkdir_p\nAlso look out for things like rake and capistrano shelling out\n\nPassing RubyStrings into java can be tricky, since they will be converted to java strings and you can get problems with string encoding. This is particulary important if the string isn’t really a string but a serialised object.\n\nFor example, Hot bunnies, which uses the rabbitMQ java library and wrapping it in a gem. When publishing data to rmq you need to serialize your data, send that string to java, which sends it across the network, then back into java and finally into ruby. \n
  • Shelling out is quite a lot slower in JRuby, so use in moderation. Most things don’t need shelling. For example this you can do with FileUtils.mkdir_p\nAlso look out for things like rake and capistrano shelling out\n\nPassing RubyStrings into java can be tricky, since they will be converted to java strings and you can get problems with string encoding. This is particulary important if the string isn’t really a string but a serialised object.\n\nFor example, Hot bunnies, which uses the rabbitMQ java library and wrapping it in a gem. When publishing data to rmq you need to serialize your data, send that string to java, which sends it across the network, then back into java and finally into ruby. \n
  • Shelling out is quite a lot slower in JRuby, so use in moderation. Most things don’t need shelling. For example this you can do with FileUtils.mkdir_p\nAlso look out for things like rake and capistrano shelling out\n\nPassing RubyStrings into java can be tricky, since they will be converted to java strings and you can get problems with string encoding. This is particulary important if the string isn’t really a string but a serialised object.\n\nFor example, Hot bunnies, which uses the rabbitMQ java library and wrapping it in a gem. When publishing data to rmq you need to serialize your data, send that string to java, which sends it across the network, then back into java and finally into ruby. \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. Gotcha! Things that will come back to bite youDavid Tollmyr, Lead Platform developer
  • 2. Show of hands
  • 3. Who?
  • 4. Who?Lead platform developer at Burt
  • 5. Who?Lead platform developer at BurtAd analytics startup
  • 6. Who?Lead platform developer at BurtAd analytics startupReal time analytics processing
  • 7. Who?Lead platform developer at BurtAd analytics startupReal time analytics processing~15k requests/s
  • 8. Who?Lead platform developer at BurtAd analytics startupReal time analytics processing~15k requests/sJRuby 1.6.x, RabbitMQ, MongoDB, Redis,Cassandra
  • 9. What?
  • 10. What?Experiences learned while building our system
  • 11. What?Experiences learned while building our systemLong running application
  • 12. What?Experiences learned while building our systemLong running applicationOne specific task
  • 13. What?Experiences learned while building our systemLong running applicationOne specific taskThousands of times per second
  • 14. What?Experiences learned while building our systemLong running applicationOne specific taskThousands of times per second
  • 15. Strings
  • 16. Strings“this creates a new string”
  • 17. Strings“this creates a new string”String.new(“this creates a newstring”)
  • 18. Strings“this creates a new string”String.new(“this creates a newstring”)1_000_000.times do hash[“string_key”] += 1end
  • 19. Stringsstring 2.107sconstant 1.730s -0.377ssymbol 1.693s -0.414sint 1.197s -0.91s
  • 20. StringsKEY = “string_key”.freeze1_000_000.times do hash[KEY] += 1endstring 2.107sconstant 1.730s -0.377ssymbol 1.693s -0.414sint 1.197s -0.91s
  • 21. Strings-verbose:gc[GC 419456K->21351K(6239040K), 0.0497610 secs][GC 440807K->31195K(6239040K), 0.0349782 secs][GC 450651K->37050K(6239040K), 0.0297697 secs][GC 456506K->26561K(6239040K), 0.0304456 secs][GC 446017K->38585K(6239040K), 0.1181651 secs][GC 458041K->29042K(6239040K), 0.0050885 secs][GC 448498K->25417K(6239040K), 0.0047236 secs][GC 444873K->24514K(6239040K), 0.0048243 secs][GC 443970K->24288K(6239040K), 0.0049824 secs][GC 443744K->24241K(6239040K), 0.0046804 secs][GC 443697K->24241K(6239040K), 0.0048907 secs][GC 443697K->24239K(6239040K), 0.0046102 secs][GC 443695K->24243K(6239040K), 0.0049074 secs][GC 443699K->24242K(6239040K), 0.0045477 secs][GC 443698K->24242K(6239040K), 0.0051179 secs][GC 443698K->24246K(6239040K), 0.0049280 secs][GC 443702K->24250K(6239040K), 0.0050470 secs][GC 443706K->24251K(6239040K), 0.0050648 secs][GC 443707K->24255K(6239040K), 0.0048519 secs][GC 443711K->24254K(6239040K), 0.0044326 secs][GC 443710K->24257K(6239040K), 0.0048402 secs][GC 443713K->24259K(6239040K), 0.0051717 secs][GC 443715K->24263K(6239040K), 0.0048119 secs][GC 443719K->24263K(6239040K), 0.0043401 secs][GC 443719K->24267K(6239040K), 0.0054948 secs][GC 443723K->24266K(6239040K), 0.0045278 secs][GC 443722K->24269K(6239040K), 0.0051420 secs][GC 443725K->24271K(6239040K), 0.0044680 secs][GC 443727K->24273K(6239040K), 0.0049075 secs][GC 443729K->24274K(6239040K), 0.0042977 secs]
  • 22. Strings
  • 23. Strings
  • 24. Array.join
  • 25. Array.joindata = [“NS”, :key, “site”, 1, nil]key = data.join(“/”)#=> “NS/key/site/1/”
  • 26. Array.joindata = [“NS”, :key, “site”, 1, nil]key = data.join(“/”)#=> “NS/key/site/1/”self calls method-----------------------------------0.35 1000 Array#join8.11 101000 BasicObject#method_missing0.07 201000 Kernel#respond_to?0.04 101000 NoMethodError#initialize0.02 100000 Symbol#to_s0.02 201000 Kernel#respond_to_missing?
  • 27. Array.joindata = [“NS”, :key, “site”, 1, nil]key = data.join(“/”)#=> “NS/key/site/1/”self calls method-----------------------------------0.35 1000 Array#join8.11 101000 BasicObject#method_missing0.07 201000 Kernel#respond_to?0.04 101000 NoMethodError#initialize0.02 100000 Symbol#to_s0.02 201000 Kernel#respond_to_missing?Array.join uses to_str, not to_s
  • 28. Array.joinJoining an array of 1 000 entries, 10 000 timesstrings 0.771ssymbols 3.171s +2.4sint 6.588s +5.817s
  • 29. Array.joinJoining an array of 1 000 entries, 10 000 timesstrings 0.771ssymbols 3.171s +2.4sint 6.588s +5.817sclass Symbol/Numeric/NilClass alias_method :to_str, :to_send
  • 30. Array.joinJoining an array of 1 000 entries, 10 000 timesstrings 0.771ssymbols 3.171s +2.4sint 6.588s +5.817sclass Symbol/Numeric/NilClass alias_method :to_str, :to_sendstrings 0.771ssymbols 1.781s +1.01s (-1.39s)int 5.484s +4.713s (-1.104s)
  • 31. Array.join Struct.new("Customer", :name, :address)customer = Struct::Customer.new("Dave", "123 Main")customer.name, customer.addressCustomer = Struct.new(:name, :address)customer = Customer.new("Dave", "123 Main")
  • 32. Array.join Struct.new("Customer", :name, :address)customer = Struct::Customer.new("Dave", "123 Main")customer.name, customer.addressCustomer = Struct.new(:name, :address)customer = Customer.new("Dave", "123 Main")NameError: identifier name needs to be constant from org/jruby/RubyStruct.java:210:in `new
  • 33. Array.joinJoining an array of 1 000 entries, 10 000 timesstrings 0.771ssymbols 3.171s +2.4sint 6.588s +5.817s
  • 34. Array.joinJoining an array of 1 000 entries, 10 000 timesstrings 0.771ssymbols 3.171s +2.4sint 6.588s +5.817sdata.map(&:to_s).join(“/”)
  • 35. Array.joinJoining an array of 1 000 entries, 10 000 timesstrings 0.771ssymbols 3.171s +2.4sint 6.588s +5.817sdata.map(&:to_s).join(“/”)strings 0.771symbols 2.242 +1.471s (-0.929s)int 5.638 +4.867s (-0.95s)
  • 36. Queues
  • 37. Queues
  • 38. QueuesInternal FIFO queue using an array
  • 39. QueuesInternal FIFO queue using an arrayPutting data with shift/push or pop/unshift
  • 40. QueuesInternal FIFO queue using an arrayPutting data with shift/push or pop/unshiftAlternatives:
  • 41. QueuesInternal FIFO queue using an arrayPutting data with shift/push or pop/unshiftAlternatives:Fixed size array with sweeping index
  • 42. QueuesInternal FIFO queue using an arrayPutting data with shift/push or pop/unshiftAlternatives:Fixed size array with sweeping indexLinkedList
  • 43. QueuesInternal FIFO queue using an arrayPutting data with shift/push or pop/unshiftAlternatives:Fixed size array with sweeping indexLinkedListLinkedBlockingQueue
  • 44. QueuesSweeping index Start End
  • 45. QueuesSweeping index Start End
  • 46. QueuesSweeping index End Start
  • 47. Queuesqueue = Queue.new(size)100_000.times do |i| data = queue.pop queue.push(i)endQueue(1_000)array 0.065sindex 0.030sLinkedList 0.056s
  • 48. Queuesqueue = Queue.new(size)100_000.times do |i| data = queue.pop queue.push(i)endQueue(1_000)array 0.065sindex 0.030sLinkedList 0.056sQueue(100_000)array 3.996s 85 times slowerindex 0.047sLinkedList 0.137s
  • 49. Queuesqueue = Queue.new(size)100_000.times do |i| data = queue.pop queue.push(i)endQueue(1_000)array 0.065sindex 0.030sLinkedList 0.056sQueue(100_000)array 3.996s 85 times slowerindex 0.047sLinkedList 0.137sQueue(1_000_000)array 53.895s 216 times slower!index 0.249sLinkedList 0.847s
  • 50. Concurrency
  • 51. Concurrency
  • 52. ConcurrencyUse a thread pool (iejava.util.concurrent.Executors)
  • 53. ConcurrencyUse a thread pool (iejava_import java.util.concurrent.Executorsexecutor = Executors.newFixedThreadPool(2)10.times do executor.submit do # do stuff endend
  • 54. ConcurrencyUse a thread pool (iejava.util.concurrent.Executors)Ruby’s Mutex? You’re doing it wrong
  • 55. ConcurrencyUse a thread pool (iejava.util.concurrent.Executors)Ruby’s Mutex? You’re doing it wrong java.util.concurrent is your friend
  • 56. ConcurrencyUse a thread pool (iejava.util.concurrent.Executors)Ruby’s Mutex? You’re doing it wrong java.util.concurrent is your friend AtomicInteger, ConcurrentHashMap, LinkedBlockingQueue
  • 57. ConcurrencyUse a thread pool (iejava.util.concurrent.Executors)Ruby’s Mutex? You’re doing it wrong java.util.concurrent is your friend AtomicInteger, ConcurrentHashMap, LinkedBlockingQueue Lock, Semaphore, CountdownLatch - IF you have to!
  • 58. Some more, just because
  • 59. Other stuff to keep in mind
  • 60. Other stuff to keep in mindShelling out is significantly slower in JRuby
  • 61. Other stuff to keep in mindShelling out is significantly slower in JRuby `mkdir -p /some/path/deep/1`
  • 62. Other stuff to keep in mindShelling out is significantly slower in JRuby `mkdir -p /some/path/deep/1` rake, capistrano, bundle exec
  • 63. Other stuff to keep in mindShelling out is significantly slower in JRuby `mkdir -p /some/path/deep/1` rake, capistrano, bundle exec There’s almost always a way of doing it without shelling
  • 64. Other stuff to keep in mindShelling out is significantly slower in JRuby `mkdir -p /some/path/deep/1` rake, capistrano, bundle exec There’s almost always a way of doing it without shellingDon’t pass binary RubyString to Java
  • 65. Other stuff to keep in mindShelling out is significantly slower in JRuby `mkdir -p /some/path/deep/1` rake, capistrano, bundle exec There’s almost always a way of doing it without shellingDon’t pass binary RubyString to Java Example: Hot_bunnies (JRuby RabbitMQ gem)
  • 66. Other stuff to keep in mindShelling out is significantly slower in JRuby `mkdir -p /some/path/deep/1` rake, capistrano, bundle exec There’s almost always a way of doing it without shellingDon’t pass binary RubyString to Java Example: Hot_bunnies (JRuby RabbitMQ gem) Serialized data -> Ruby -> Java -> Network -> Java -> Ruby
  • 67. Other stuff to keep in mindShelling out is significantly slower in JRuby `mkdir -p /some/path/deep/1` rake, capistrano, bundle exec There’s almost always a way of doing it without shellingDon’t pass binary RubyString to Java Example: Hot_bunnies (JRuby RabbitMQ gem) Serialized data -> Ruby -> Java -> Network -> Java -> Ruby String#to_java_bytes / String.from_java_bytes
  • 68. Conclusion
  • 69. Conclusion
  • 70. ConclusionBeware of unnecessary string creation
  • 71. ConclusionBeware of unnecessary string creationWatch out for array.join with non-string objects
  • 72. ConclusionBeware of unnecessary string creationWatch out for array.join with non-string objectsDon’t shift/unshift large arrays
  • 73. ConclusionBeware of unnecessary string creationWatch out for array.join with non-string objectsDon’t shift/unshift large arraysjava.util.concurrent is your friend!
  • 74. ConclusionBeware of unnecessary string creationWatch out for array.join with non-string objectsDon’t shift/unshift large arraysjava.util.concurrent is your friend!Don’t shell if you don’t have to
  • 75. ConclusionBeware of unnecessary string creationWatch out for array.join with non-string objectsDon’t shift/unshift large arraysjava.util.concurrent is your friend!Don’t shell if you don’t have toRemember String#to_java_bytes
  • 76. We’re hiring!david@burtcorp.com @effata burtcorp.com
  • 77. Thank youdavid@burtcorp.com @effata burtcorp.com