We've noticed a lot of people are getting frustrated at the apparent lack of updates on player pages and the like, so I thought i'd give an update of just what changes we've been doing behind the scenes. We've made considerable changes to the backed, so i'll go through each of them, in the order we've tackled them.

Batch Processing

We've been working hard trying to scale up the database to handle the huge increase in players we track. We are close to 18.5 million characters, and that's rising every day. While this isn't a huge amount of data, having so many records makes finding an individual character rank slow. While we've indexed our data, these only help in certain situations. So while getting a list of the top 100 players in the US is super speedy, finding the US rank for player XYZ is slow. And it gets slower the further down in rank you are.

To get around this problem, we calculate an ordered list of rankings every day for regions and world. To keep the server responsive, we only calculate the top 1,000,000 players for each rank type (ie, the top 1,000,000 world players, and the top 1,000,000 for each region). We may increase that number, depending on if you guys really want a rank going into the millions :) We are also considering other solutions to speed it up (anyone used MongoDB for range queries and data analytics? let me know how it went), but those are quite large and involved.

Site responsiveness

We've tried to optimise all database queries as much as possible, but there are still some queries that will take more than 5seconds to complete. Most of these are for relatively static content. A rankings page, for example, is only going to change once the batch job updates, so its useless regenerating that content each time someone goes to the page. We've enabled static caching on all ranking and player pages so viewing them will be fast, but they will only change after the batch update has run.

We've also moved all our css/js/images onto Amazon Cloudfront CDN, so loading images etc should be faster for everyone. The site is now running on nginx instead of Apache, and there is a memcache daemon running as well for a couple things.

Player Updates

There have been a lot of issues with this recently. Most of the scanners/updaters have not been running as we scramble to fix other issues with the site (you may have noticed its been down a lot over the last few weeks). When we took over the site, we made a change from having the 'update now' button to a different scheme of automatically inserting the character into a priority queue when you visit the character page. This queue is currently sitting at ~250,000 characters.

It would be great to get a huge update from the Armory and update everyone at once, but unfortunately we have to update each character individually (if anyone at Blizzard is reading this, we'd love to chat re: Armory updating). If updating 250,000 players isn't refreshing player pages fast enough we'll try some other things, but since the scanners have been dead more often than not over the last couple weeks, we'll give them a little time to catch up.

Hopefully this has given you an idea of what we've done behind the scenes and that we are continually improving the site. If you have any questions/comments, let us know in the comments