[pmwiki-users] pagelists link= with pages generated externally

[Patrick R. Michaud]

On Mon, May 28, 2007 at 10:24:14AM +0200, Christophe David wrote:
> >If time expires before import has finished processing files, it attempts
> >to change the timestamp of the .lastimport file so that it will continue
> >processing more imports on the next browse request.  (Regardless, it will
> >check things again after $ImportFreq seconds have elapsed.)
> 
> This does probably not contribute to the expected result: when you
> have say 3.000 pages to import (small ones with only a few
> PagetextVariables ;-) ), all visitors (or always the same visitor if
> there is only one active) will have all their requests taking 15
> seconds, and the CPU will be loaded as is there was no "regulation"
> mechanism, possibly triggering alarms and/or problems with the hosting
> company.

This isn't entirely accurate.  The visitors never see their requests
taking 15 seconds, as the pages are updated in the background _after_
each visitor's normal request is handled.  In other words, visitors
never see a delay in response time due to the import, unless they know
exactly what to look for.

However, I've also been thinking that there should be an $ImportDelay
parameter, which says that any given file (including .lastimport)
has to be at least $ImportDelay seconds old before the script will
process it.  The default would be something like ten seconds.  This 
will help avoid processing files that are currently being written 
by an external process.

If one then sets $ImportDelay to 15, and leaves $ImportTime set at
15, the result would be that the system would spend at most 30
seconds out of each minute processing imports.  It also means
that $ImportFreq could be set at a very high number, which avoids
frequent scans of the import/ directory even when there's nothing
new to import.

Pm