Post Tagged with: "Heroku"

One of the problems I have often had with caching is that it becomes more difficult the second you grow past a single server – and let’s face it, most sites these days either aren’t or shouldn’t be running on one box.

I’m a huge fan of Heroku, and could talk for a long time about how their service inspires developers to write things by taking away almost all of the complexity of actually running your creation. The downside of this is that you’re left with some expensive options for basic tasks. Whilst 5mb of Memcached is free, 100mb is $20/mo and 250mb is $40/mo. MongoDB offers a lot more storage for low/free cost – and is pretty fast to boot. Coupled with the fact that it is a persistent data-store, it is ideal for storing your cache entities in – these are the things that we don’t really want to have to rebuild very often, and when we scale a new dyno up, having access to essentially a warm cache right off the bat is ideal. There are, of course, a lot of SaaS Mongo providers out there, but you can get up to 240mb of storage for free…

To plug this easily into Rack::Cache, I’ve written a patch which supports MongoDB entitystores. I did not code the MetaStore functionality, as I believe that is better served from Redis or Memcached – which is likely to be a bit quicker. There is a pull request for this to be brought into Ryan Tomayko’s master, but until (if) that happens, you can get it from the “mongodb” branch at http://github.com/jgwmaxwell/rack-cache. To use this in an app, you can put the following in your Gemfile:

then to configure the MongoDB EntityStore, you can do one of 2 things where you configure the middleware:

With this you’re ready to go. Hope you find it useful, we are using it in production at Cambelt and it hasn’t missed a beat so far, with speed slightly faster than using the Disk EntityStore.

One of the cool things about Heroku is that it makes life really easy for those who use it – and for 99% of applications I love being able to simply forget about sysadmin tasks and just get on with writing code. Lots of people have started using Unicorn to serve apps on Heroku, which allows you to fork off worker processes to absolutely maximise the computational power of each Dyno. There have been some great articles on this, such as this one on using Unicorn and this one on using resque workers in-dyno. This certainly can give you a performance boost, but I’ve never been a massive fan of Unicorn, much preferring Thin. I’m not going to go into a full discussion of Thin vs Unicorn and the relative merits of each system, however normally the discussion is a Unicorn system with 3/4+ workers, vs a single instance of Thin per Read more…

Collecting Data through an API I’ve been working on improving an API that collects stats for an application I run. Initially we were collecting data through Rails, as the rest of the application runs on rails, but I started thinking that this was surely overkill – after all this is simply stats data – what is needed is realtime, highly concurrent collection – which the advanced features of rails are not particularly suited to. Let’s face it – a packet describing a particular stat is unlikely to be wrongly formed, and the business consequences of saving a malformed packet are low in the extreme, and a background process can quite happily walk through the database removing invalid packages when the service is quiet – allowing eventual validity to occur. So, I started to re-factor, and more importantly re-think. A stat packet, collected either by a HTTP request – such as for Read more…