Quick Benchmarks of Ruby Interpreters

I was curious about how the performance of the new MRI 1.9.3 Interpreter compares to 1.9.2, so I threw together a very quick test application using Sinatra::Synchrony, and tested it under the two Interpreters, as well as Rubinius. I only haven’t tested with JRuby due to the difficulty in getting the fiber aware code for Rack-Fiber-Pool working with something other than WEBrick, and I’m not cruel enough to publish tests using that!

I ran the same test at various levels of concurrency, so the following graphs give you what came out.

Concurrency 10

Concurrency 20

Concurrency 50

Concurrency 100

Concurrency 250

Well, this is interesting, and what I’d hoped to see – MRI 1.9.3p0 is consistently faster than 1.9.2p290 across the board. It would be a sad day if it wasn’t – let’s face it – we are hoping for progress with each iteration, but good to see.

I have to say I was amazed Rubinius performed so poorly. Possibly it was because it was due to being in 1.9 mode, which is still experimental. On digging deeper, the instances of Thin were running up to 500meg of RAM at the end of these tests on Rubinius, but only 30 on MRI, which suggests that Rubinius was having real issues handling either the Threads or Fibers. I’ll look into this and repost a score for Rubinius when I’ve got it sorted, certainly from my experience I’ve found Rubinius to be a very fast VM.

So, the price of progress appears to be more speed, which is no bad thing at all…

The app used to run the tests can be found here on github, for anyone who is interested. The machine is just my travel laptop, so a little underpowered (Core i3, 8gig Ram, SSD) – better results from production machines should be easily possible.