Here is a short video clip of a few of the always running processes on the Politwitter server during the french language leaders debate.

Along with a few processes that are running 24/7 there are 40 or so tasks that run in the background on a periodic basis to keep the site updated, refresh stats along with other things. Doing all this processing in the background allows the website to load fast, which has always been a priority from day one.

Politwiter also caches the output of a lot of content, like the sidebar blocks, many are cached anywhere from 1 minute to 30 minutes. There are over 11,000 files in the cached content folder right now. Gzip is also used to speed up delivery.

Politwitter is run on a Xeon Quad core Dell Poweredge server running Windows Web Server 2008. A second Dell poweredge server is used as a dedicated database server. Bother servers have 4GB of ram, which is used to give MySQL plenty of room for in-memory caching. Politwitter is run on IIS7 using PHP. The website is custom coded by Trevor May and doens't use any 3rd party frameword or CMS. The uses jquery along with other 3rd party platfrms like Goole Visualization API for charts.

That's all for now, I might write a post detailing some of the ways Politwitter deals with the large amounts of data and how the Twitter API's are used.