[pmwiki-users] How to be sure pageindex is being used for pagelists (using debug messages ?)

ABClf languefrancaise at gmail.com
Sun Jan 10 17:23:55 CST 2016

Not easy to say because I don't have access to all relevant
informations (shared hosting) and because all (or almost all) pages
are using pagelists (at least basic PL in the sidebar).
All its working fine now (not blazing fast yet quite usable for me)  ;
I experienced unusual very slow processing yesterday (where it is
usually fine), and the day before, for part of the day only.
I opened this thread to ask how to be sure pageindex is truly used
(ie, not corrupted and not ignored for example).
Because I'm on a shared host, I don't have information about what's
happening in the server (disk usage, files active, memory, etc., i'm

Here is an hypothesis :

In my setup, I have a lot of (small) pages (around 200k, slowly
growing, often edited), most of them using pagelists for agregating
content (70k words are connected to 100k quotes).
The pagelists used are not complex, but it's true they have to collect
small pieces of data among many pages (some do use ptv). It works
decently most of the time for me (almost all visitors are anonymous
and can enjoy the already-existing cached files).
When I want to collect synonyms for a word, it takes about 10 seconds
(tested just now, when status is fine), to build the result page. Much
much more faster if cached. Say, in a short time, I want to open 3
uncached pages, in 3 new windows, Synonyme.1, Synonyme.2, Synonyme.3,
it takes very very long until I get the 3 result pages done in 3
I might say the server is not powerful enough (or maybe it can't
handle the pageindex in such a short time).

To make this work for non logged visitors, despite that limitation, I
use fastcache recipe, which is a must have recipe for me.

There is one issue with fastcache though : it can delete all, or
nothing, every time a page is updated, but there is no setup for
deleting the page according to its age. Because I obviously don't want
to delete 70k cached pages every time I update one page in the website
(in that case, website would become unusable for a week, always busy
building cached pages, until most of the pages are cached), I set it
to delete nothing.
And because I need to have oldest pages rebuilt , I manually delete
5000 oldest cached pages every week or so, with that command
(terminal) :
ls -1tr | head -5000 | xargs -d '\n' rm -f

Then, pmwiki, with fastcache, has to rebuild every deleted page every
time a new visitor is visiting it. Some take 10 seconds, or less, or
My hypothesis : it is possible my website has become slow these days
because the server was busy rebuilding the cached pages which have
been mass deleted last time.

I won't mass delete cached this week, to see what happens.

Thank you for your contribution Peter.


If anyone is interested in testing fastcache recipe, here is a pagelist :

2016-01-10 21:18 GMT+01:00 Peter Bowers <pbowers at pobox.com>:
> Are you seeing this performance on all pages or only those with pagelists?
> Be aware that some pagelist search requirements require an exhaustive search
> through files rather than using the pageindex (searching for the value of a
> PTV, for instance).
> -Peter
> On Sat, Jan 9, 2016 at 4:57 PM, ABClf <languefrancaise at gmail.com> wrote:
>> Thank you for your clean summary Petko.
>> I'm asking that question because my website has become very slow (30
>> seconds or more, many 502 bad gateway, no reason I can point. Not sure
>> it is my fault or server's fault).
>> According to the data given by diag option, it looks like the scan
>> among all my folders and files took too much time.
>> Thus I suspected the existing pageindex (57,2 mo) to have become
>> altered and useless...
>> I built a new one, and it's still very bad. I must now ask my webhost,
>> and start investigating for a new one.
>> Gilles.
>> 2016-01-09 8:26 GMT+01:00 Petko Yotov <5ko at 5ko.fr>:
>> > On 2016-01-09 00:05, ABClf wrote:
>> >>
>> >> For new year first question, I would like to know how to be sure
>> >> .pageindex is used for creating pages. Do informations given by
>> >> $EnableDiag help ?
>> >
>> >
>> > For creating pages? .pageindex is used (scanned) when a pagelist or a
>> > visitor searches for terms or link targets in the wiki pages. When
>> > searching, you will see something like:
>> >
>> > 16: 00.17 00.16 PageListTermsTargets begin count=1038
>> > 17: 00.17 00.16 PageIndexGrep begin
>> > 18: 00.28 00.27 PageIndexGrep end
>> > 19: 00.96 00.94 PageListTermsTargets end count=222
>> >
>> > The "PageIndexGrep" lines indicate that .pageindex was used.
>> >
>> >> (For example : does this page
>> >> http://www.pmwiki.org/wiki/Test/PagelistExample use pageindex or no ?)
>> >
>> >
>> > No, this page does not use .pageindex because the pagelist does not need
>> > to
>> > search for terms or link targets.
>> >
>> > Moreover, we use $PageListCacheDir so the whole list of pages from the
>> > pagelist will be saved once and as long as the cache is not obsolete, it
>> > will use the cache without scanning the pages or .pageindex.
>> >
>> >
>> > If you wonder if .pagelist is updated with the terms and targets of a
>> > newly
>> > created or a modified page, the answer is yes. To see it in the
>> > diagnostic
>> > output, disable redirects ($EnableRedirect=0;) then when you save a
>> > page,
>> > you will see these two lines:
>> >
>> > 30: 00.38 00.35 UpdatePage: PostPageIndex (Test.WikiSandbox)
>> > 31: 00.38 00.35 PageIndexQueueUpdate: queued 1 pages (1 total)
>> >
>> > try it here: http://www.pmwiki.org/wiki/Test/WikiSandbox (you can delete
>> > and
>> > recreate this page if you want).
>> >
>> > ---
>> > Change log     :  http://www.pmwiki.org/wiki/PmWiki/ChangeLog
>> > Release notes  :  http://www.pmwiki.org/wiki/PmWiki/ReleaseNotes
>> > If you upgrade :  http://www.pmwiki.org/wiki/PmWiki/Upgrades
>> >
>> >
>> > _______________________________________________
>> > pmwiki-users mailing list
>> > pmwiki-users at pmichaud.com
>> > http://www.pmichaud.com/mailman/listinfo/pmwiki-users
>> _______________________________________________
>> pmwiki-users mailing list
>> pmwiki-users at pmichaud.com
>> http://www.pmichaud.com/mailman/listinfo/pmwiki-users

More information about the pmwiki-users mailing list