[pmwiki-users] RFC: Ratings Redux

[Joachim Durchholz]

Ben Wilson schrieb:
First, I think a five-point scale is
> easiest to wrap the mind around (1-dislike, 2-somewhat dislike,
> 3-neutral, 4-somewhat like, 5-like).

Good proposal.

> Second, I think a rating on two-axis is preferable (or three). I'm not
> sure of what to attribute to each axis. However, a scale on the
> recipe's ability to do as advertised helps, as may another on
> complexity (which may really be better addressed in the
> documentation). So, the second issue is how many axes and what are
> they rating.

The problem with "vitality" etc. is that this is all quite difficult to 
rate. Automatic rating systems tend to measure nonsense, and manual 
rating is simply too annoying, only very few people have the time and 
diligence to do a quality rating. (And we don't want to rate the ratings...)

How about more visible qualities? E.g. axes like:
* Overall quality
* Installation effort (including hassles)
* Documentation completeness
* Ease of understanding (docs)
* Security relevance
Several of these ratings could also be filled in by the recipe maintainer.

> Third, I think an actual opinion helps.

Textual opinions are already accounted for. Simply add a comment to the 
recipe page.
If the rating indicators are just PmWiki markup, one can even add 
comments immediately below the rating. Or whatever works best.

> Popularity may be possible in other ways.

Another way: count the total number of ratings given - that one would be 
automatic and reasonably accurate.

A third way: have something similar to Debian's popularity contest tool. 
Installed recipes will report their activity to a central site.
That would be a very automatic way to get at reasonably reliable 
deployment data, and help assess which recipes need most attention e.g. 
when updating PmWiki.
Potential liabilities that I see:
* Configuration-only recipes won't show up on the radar.
* PmWiki must offer a global switch to disable contest votes.
   We don't want to send data without the administrator's consent.
* A server must be set up to collect the votes.
   (Probably a script on PmWiki.)
* Reporting needs to be limited to, say, once per day.
   We don't want to hammer the votetaking server once per PmWiki request
   world-wide. Or the recipe could collect usage data over a period of
   time (hours/days/weeks/whatever) and transmit accumulated statistics.

Regards,
Jo