[pmwiki-users] Wiki speed diferences, no apparent reason
Patrick R. Michaud
pmichaud at pobox.com
Wed Mar 22 08:08:45 CST 2006
On Wed, Mar 22, 2006 at 02:09:05PM +0100, Joachim Durchholz wrote:
> Philip Stitt schrieb:
> > Sometimes my website is unbearably slow. It will get that way for about
> > 20 to 30 minutes at a time. If I spend all day working on my site, I'll
> > see this happen maybe 5 or 6 times during the day, sometimes more.
> I have a similar phenomenon. PmWiki pages are sometimes really, really slow.
> I don't know whether that's Apache, FastCgi, or PmWiki.
Over the past few months I've been doing a *lot* of investigation
into why things get slow, and I still don't have any clear answers.
Here are some of my observations thus far:
* It almost has to be related to resource contention of some sort,
as opposed to PmWiki being inherently slow. As many have observed,
there are times when PmWiki does run quite quickly, which means
that its internal algorithms aren't intrinsically slow.
* pmwiki.org seems to slow down during the middle of the day
(U.S. timezones), when there are a larger number of incoming
requests. This also points to a resource contention issue,
although I can't be certain if it's PmWiki that is eating up
the resources or some other application(s) on the same server.
* I think Apache has a bug somewhere that prevents it from
recognizing dropped connections (and it seems to spinlock).
I've never been able to duplicate it on my own, but occasionally
when I do an Apache server-status request I'll find processes
that have been running for 5 minutes or more. Often these are
PmWiki processes, but sometimes they're simple file requests
that seem to have been "hung" somehow.
I know that for many of the PmWiki processes that seem to be
taking a long time, it's because they're waiting to send output
and not because they're actively processing content.
I also find it bizarre that many of the PmWiki/PHP processes
continue running beyond the "max execution time" parameter
(30 seconds on pmwiki.org), which again would seem to indicate
a problem in PHP or Apache and not PmWiki.
* Until a few days ago, pmwiki.org's performance was extremely
sensitive to incoming messages to the pmwiki-users mailing list.
My server currently uses mailman for mailing lists and
(unfortunately) exim for mail transport, and both of those
packages tend to be disk and memory intensive. And mailman
doesn't have strong support for throttling traffic. So, an incoming
message to pmwiki-users would rapidly spawn a few dozen exim
processes that would instantly spike the server load and take
a long time to complete. Of course, as they take longer to
complete, it becomes more likely that another message will arrive
and spike things even further.
Late last week I was finally able to find the mailman/exim settings
needed so that these spikes don't happen, and performance has been
greatly improved since then.
* My first inclination has been to think that the primary bottleneck
is memory, since PHP (and PmWiki) can use up a lot of memory while
running. Unfortunately, PHP doesn't have good facilities for
monitoring memory usage, so I haven't been able to confirm this
theory. However, I recently upgraded the RAM profile on my
virtual server, and pmwiki.org still experiences the occasional
slowdown, and AFAICT I'm not using all of my available memory
* Which leaves us with disk contention as the primary candidate for
slowness -- i.e., the operating system is slow in responding
to requests to open or read files. I suspect this can be a real
issue in shared hosting or virtual server environments, where
there are many processes all trying to access disk resources
in one way or another.
* Anecdotally, when PmWiki is running slow I also seem to have
high disk latency from my shell command prompts -- commands that
don't require any disk accesses will run extremely fast, but
there's a definite lag in any command that requires opening or
reading a file of some sort.
> Well, for the moment, the best would be some kind of log that writes
> PmWiki's processing times to some kind of CSV file.
I've been thinking I need to develop a diagnostic log for this
to run on pmwiki.org, but I hadn't thought about doing it as a CSV file.
That's a really good idea.
I think the easy way to approach it will be as a wrapper script
to pmwiki.php -- i.e., something that can grab the relevant variables
and save them to a log file. Either that or I'll find a way to
log PmWiki's internal stopwatch (PmWiki has a "StopWatch" diagnostic
function that I use to bench mark things and find out where things
> That way, one can
> see whether the lags are due to PmWiki or outside, and if they are
> within PmWiki, we might be able to correlate the slow requests with
> specific pages, or specific query strings, or various kinds of system
> load (memory/CPU load, number of active Apache processes, and whatnot).
I've looked for a long time and haven't found a correlation to
specific pages, query strings, or requests. I'm pretty sure it's
more system related, i.e., with Apache, PHP, or the OS.
More information about the pmwiki-users