[pmwiki-users] SQLite pagelist recipe prerelease

[Alex Eftimiades]

Actually, I do not have a single pagetextvariable involved in the  
searching. I do have an if="before $date" (with custom conditional for  
"before" and a custom page variable $date). Those however are not  
found within the text.

I suspect the holdup is in FmtPageName for order=group to resolve the  
group based on pagename. I was looking at the FmtPageName code, and it  
sometimes reads the pages, or passes them on to PageVar which  
sometimes reads pages as well. This means that whether I use the  
database pagelist recipe or not, I am still reading each page for  
certain situations.

I will look into those other options you mentioned, and possibly a  
more efficient way to generate the pagevariable data. The advantage I  
thought I was achieving was mainly cutting down on the number of pages  
to sift though when the pagelist first generates a list of every  
single page on the wiki. I thought that using the database to create a  
smaller list of matches to several criteria would give me a a great  
advantage. Again, maybe this only happens at absurdly large numbers of  
pages.

I have actually never heard of the databases you mentioned (htDig and  
nepomuk). I will look into those if you think that would give me the  
speed boost I am looking for--although you seemed to base you claim on  
the (incorrect) assumption that I was doing lots of pagetextvariable  
searching.

Anyway, thanks for your input, and please let me know if you come up  
with anything else. Any and all comments are appreciated.

Thanks,
Alex

On Aug 26, 2012, at 4:02 PM, Christopher Cox wrote:

> Free text word searching (which is probably behind the OP's  
> performance and pagelist, etc issue).. is also something better  
> suited to a dictionary word search db thing... stuff like htDig did  
> for static HTML... things like nepomuk also come to mind.  So.. I  
> think you may need something that creates a high speed lookup into  
> the flat files for free text searches maybe?
>
> (afterthought to my prior post)
>
> On 08/26/2012 02:58 PM, Christopher Cox wrote:
>> Can't say for sure.  But figure that PTVs (for example) are just  
>> (more or less) name,value pairs.  So... an RDMS unless there are  
>> clear benefits to be had using some kind of relationship joins  
>> across tables... well... it's likely the wrong tool for the job in  
>> many cases.  Not saying that you might have been PTV bound here....  
>> just saying... PmWiki does come with a pagelist cache mechanism  
>> (which can help in some cases).
>>
>> So... at least for me, the concern would be more focused on users  
>> of lots of PTVs and how that might impact performance if the number  
>> of pages got really, really large.  In those cases, a name,value db  
>> might help.
>>
>> If you believe that there are many strong row/columns style RDBMS  
>> relations that somehow fit into the operation of PmWiki, then maybe  
>> a RDBMS fits...  I'm thinking it's the wrong tool for the job at  
>> the moment though.
>>
>> You have to remember, things like sqlite are built on top of the  
>> same (performance wise) file system as the (sort of) indexed flat  
>> files that sit behind PmWiki.  It wouldn't surprise me if used for  
>> page fetches and stores that sqlite is actually slower.
>>
>> Just saying... (these are however, very, very cursory thoughts...  
>> somebody else probably KNOWS better)
>>
>>
>> On 08/26/2012 02:48 PM, Alex Eftimiades wrote:
>>> I have been trying to redesign some of the common pagelist  
>>> functionality to take advantage of an sqlite installation. You  
>>> just include it after you set your WikiDir to the custom SQLite  
>>> pagestore class. Like the person who sent out the updated pmwiki  
>>> skin, I was looking for some feedback/comments on the  
>>> implementation and whether you think this might help speed up  
>>> pagelists on an sqlite installation with a lot of pages.
>>>
>>> This actually has not helped speed up my own pagelists as much as  
>>> I thought it would. In fact, I do not really notice a difference  
>>> at all! Then again, I only have about 550 pages on my site.  
>>> Perhaps I would notice more of a difference if I had several  
>>> hundred thousand. All the more reason I wanted some feedback on  
>>> the implementation. Right now, it takes care of the name=, group=,  
>>> link=, and order= statements using SQLite queries. I hope to  
>>> extend it to do the page (text) variable specifications and if  
>>> statements as well.
>>>
>>> It uses a few custom functions to do FmtPageName for less standard  
>>> order=$Name type declarations and to check if the link= is in the  
>>> targets of the current page it is looking at. I would assume that  
>>> would make the sqlite query closer to just using php, but I  
>>> thought it would make a difference doing the testing on each page  
>>> as they are read from the database rather than reading them all  
>>> from the database into memory and sifting through the results from  
>>> there. I could be wrong, and this might not help at all, or  
>>> perhaps only help in certain situations.
>>>
>>> Let me know if you have any insight on this.
>>>
>>> Thanks,
>>> Alex
>>>
>>>
>>>
>>> _______________________________________________
>>> pmwiki-users mailing list
>>> pmwiki-users at pmichaud.com
>>> http://www.pmichaud.com/mailman/listinfo/pmwiki-users
>>
>>
>> _______________________________________________
>> pmwiki-users mailing list
>> pmwiki-users at pmichaud.com
>> http://www.pmichaud.com/mailman/listinfo/pmwiki-users
>
>
> _______________________________________________
> pmwiki-users mailing list
> pmwiki-users at pmichaud.com
> http://www.pmichaud.com/mailman/listinfo/pmwiki-users