nyingen (22:24:25)
*permutation

nyingen (22:24:05)
the crawler attack was hitting every combination of tags in the tag cloud

nyingen (22:23:42)
DB load, not HTTP server load, seemed to be the problem

zoi (21:52:22)
queue page is probably slow, too, but I guess better odds of hitting cache

zoi (21:51:29)
like new uploads, user favourites, songs by cyborg jeff or SIDs.

zoi (21:50:07)
seems those slow pages that list songs like to tie up the limited number of processes on the webserver

mirrorbird (21:48:39)
less efficient tho, they have to pull the pages to see it

mirrorbird (21:48:11)
(ah just noticed a 'meta noindex' tag so maybe it's being done that way per page)

mirrorbird (21:47:23)
no robots.txt file -- do we really want the (well-behaved) bots to grab every page? some like the last x hours of oneliner seem a bit pointless to put in search engines

HexaDecimator (21:30:23)
I can feel them crawling in my skin

Global Trash 3 v2

Jesper Kyd

AlcAsh

Time Left: 11:06