[pmwiki-users] (no subject)

[Patrick R. Michaud]

On Fri, Sep 08, 2006 at 07:08:00PM +0200, Tom Lederer wrote:
> Hi all,
> 
> all of the sudden i noticed that no search results are found.
> 
> Just upgraded to 2.1.25 and was about to correct any UTF-8 error that  
> come in wikilib.d pages.

There's a fundamental problem with character set encoding in
wiki pages, that hasn't been adequately resolved yet.  PmWiki doesn't
(yet?) know how to convert between utf-8 and other character sets,
so PmWiki just uses whatever character set was active when the
page was created.

The people who authored the PmWikiDe.* pages chose to stay
with iso-8859-1 (Latin-1), so thats the way those pages are
encoded.  If a site chooses to instead use utf-8, then the
distributed PmWikiDe.* pages will no longer display properly 
a site configured as utf-8.

We can't just convert all of the PmWikiDe.* pages to utf-8,
because that would then break any sites that are using iso-8859-1.

And we can't just switch everyone over to utf-8, because the
preg_* functions in PHP that perform the bulk of PmWiki's
translation work don't understand utf-8.  In particular, the
functions can't sort utf-8, and they don't understand uppercase
and lowercase mappings in utf-8, so we can't do anything
requires converting case or case-insensitive comparisons
(both of which are also desirable in PmWiki).

PHP does have an optional mb_string module, but lots of webhosting
services don't enable mb_string along with PHP, so we can't rely
on that.

Anyway, in the background I've been coming up with possible ways
to make utf-8 more workable, or at least to get the iso-8859-1
pages usable in a utf-8 environment, but so far there's no magic
bullet to resolving things.

> I do not know if the missing search results are a new "feature",  
> because i do not search often on my own pages.
> Here is the URL: http://www.celok.de/Inhalt/Suche?q=%2F&%23
> 
> It is german, hope that isn't a problem, and if i am logged in all  
> works fine. (all 7 result Pages are shown).
> 
> Any clue where to look?

If you're saying that search works when you're logged in
but doesn't work when you're not, that seems to imply that
something is odd with page permissions (perhaps a read password
on the Site group) and not with the encoding.

Hope this helps,

Pm