This post originated from an RSS feed registered with Python Buzz
by Phillip Pearson.
Original Post: www.myelin.co.nz performance testing
Feed Title: Second p0st
Feed URL: http://www.myelin.co.nz/post/rss.xml
Feed Description: Tech notes and web hackery from the guy that brought you bzero, Python Community Server, the Blogging Ecosystem and the Internet Topic Exchange
My main web server has been pretty slow for ages. I was pretty sure that either the Topic Exchange or the Python Community Server was to blame, but didn't know which, or why.
After a whole heap of twiddling - shutting down various bits and pieces until things sped up, then putting them back in one by one - it seems that BOTH sites were running slowly, basically because of the huge amount of spam being submitted to each.
I've just disabled the comment script on pycs.net, and that made it take about 1/10 the memory and run much quicker. I've also disabled posting new items to the Topic Exchange, which I doubt will hurt anyone as I don't think it has any users any more - it's all just spammers and search engines.
Interesting to see actually HOW MUCH load was being put on the system by just the Topic Exchange. 'iostat 1' shows the system reading 1500-4000 kB/sec from the hard disk when the ITE is up, and this drops down to below 100 kB/sec when I take it down (except for the occasional peak of ~2000 kB/sec for 3 s or so).
Perhaps it's time to retire the Topic Exchange. I was thinking of resurrecting it web 2.0 style (with everything requiring a login, compared to the old way of leaving access totally open), but is there really a need for another post tagging/sharing system?
That's kind of a sad thought - as the Topic Exchange was the *first* medium-scale post tagging system. (Closely followed by k-collector). I'll see if I can get rid of some of the spam in the database, which should speed it up to the point where I can afford to run it in read-only mode indefinitely.
Previous Topic
