[pmwiki-users] pmwiki.org performance
Patrick R. Michaud
pmichaud at pobox.com
Wed Mar 15 13:45:51 CST 2006
On Wed, Mar 15, 2006 at 06:56:14PM +0100, Sebastian Pipping wrote:
> At the moment surfing pmwiki.org is significantly slower
> than it was a few (say two) days ago. Am I the only one
> with this impression?
A couple of days ago my webhosting provider "upgraded" the software
and kernel that is running my virtual private server.
It was fine the first day, but since then I've had a lot of
issues with server load, and I'm still tracking them down. I
do know the following:
- mailman and exim on my server don't seem to be tuned very well,
because each message to the pmwiki-users list generates
dozens of exim processes, which instantly drive up the
server load by quite a bit. Exim in particular doesn't
seem to be able the amount of traffic we generate.
- One of either mailman or exim was having trouble with locking
of some sort, which was also increasing the load.
- The site is being hammered by search engine spiders.
Here are stats for March 10th to the present:
Total number of hits to pmwiki.org : 457,339 (100.0%)
Hits coming from Slurp (Yahoo!) : 84,008 ( 18.4%)
Hits coming from Googlebot : 30,945 ( 6.8%)
Hits coming from msnbot : 18,615 ( 4.1%)
In short, about 30% of the traffic coming to pmwiki.org
is coming from search engine robots, and the sheer numbers
of hits are starting to have an effect. I'm even considering
some judicious blocks on Yahoo!'s spiders.
More information about the pmwiki-users