[pmwiki-devel] New Recipe: Content

[Martin Fick]

On Friday 08 June 2007 06:52:44 pm Crisses wrote:
> On Jun 3, 2007, at 3:53 AM, Martin Fick wrote:
> > I would like to announce the new Content recipe:
> >
> >   http://www.pmwiki.org/wiki/Cookbook/Content
> >
> > This recipe is an API and is not useful to you unless:  you are a
> > recipe developer or you use another recipe which requires this recipe.

...
> "Content" is not descriptive enough.  

Fair enough. :)  I am open to suggestions, it is a little difficult since it 
is an API.  Maybe it should have been ContentAPI?

> Can you list several ways you can see this being used?  You don't have to
> write the recipes, but  I'm sure you have a vision for what it could do,
> hence why you think other people can use it :)

A) This recipe's primary strength is for embedding images in wiki pages that 
are generated from source text embedded in a wiki page, particularly if 
several format conversions are needed to create the image.  This is where I 
hope other people will take advantage of it most, I know there are much more 
creative people out there than me who could come up with neat dynamic images 
to create from wiki pages or form submissions.  :)  

- I would like to create a graphing utility:

    Provide a content type that can be input to some free software 
   graphing utility such as gnuplot.  The output would get embedded as 
   a graph in the current page.  *1

- Maybe a fancy font text generator, remember the 90s when people used gifs to 
create custom fonts? :)

- An electrical circuit (schematic) illustrator

- Game illustrators, for example a soccer coaching tool that displays field 
layouts.  Maybe even a billiards illustrator.

- A class file compiler for coding java applets directly in a wiki page.  
eeevil ;) 



B) Secondly it is very useful for exporting data from wiki text into various 
formats.  There are several recipes that do that already and it would be easy 
to integrate some of those recipes with this API giving authors greater 
control of which data is used to create the alternative content form.

> Example: would this be helpful to the wiki->pdf recipes?

As you mentioned, the PDF export recipe would be a good candidate for this.  
With the content API this could be enhanced so that individual sections of 
pages could be used to create PDFs instead of only entire pages.  But another 
enhancement would be to easily create other export types from the PDF, 
perhaps converting the PDF to PS.


C) This recipe could also be used to simply supply a link (without any 
conversions even) to raw text sections embedded in a page.  CSV example 
below:

> Is this mainly wikidata->output formats or does it also include data-
> wikidata input functionality?

It is mainly wikidata output formats, no input (import?) functionality.  It 
does have retrieval methods that could be used to insert output types derived 
from input types into a page, but that is not quite an import.  So,  for 
example, one of the recipes that I am writing will be a "Simple Table" to CSV 
converter.  This recipe will allow authors to surround a pmwiki simple table 
with the content directives and automatically get a link below the table 
which provides the data in the table as a CSV file.  

Conceptually this could also be reversed though, a recipe could be written 
that allows CSV data to be written in a wiki page and surrounded by a csv 
directive which would then create a CSV link for easy CSV downloading.  In 
turn a CSV to "Simple Table" converter could be written and then the 
ContentGet() function could be used to embedd the output of this converter 
into a wiki page.   I have written some markup functions to make this 
embedding easier but I have not released them yet since I want to first 
create a recipe that uses them so that I get a better feel for their correct 
API.

Admittetly if you just wanted a markup to turn CSV into "Simple Tables" you 
would be better off just creating a markup rule to do that.  The reason that 
you might opt instead to use some of the content mechanisms to do this is if 
you want easy export functionality of that CSV, or if you also wanted to use 
the CSV data in other ways, for example to create a graph.


Some other existing recipes that I think could benefit from the same 
advantages, some more than others (this is a rough guess, these may or may 
not be appropriate):

  http://www.pmwiki.org/wiki/Cookbook/GeneratePDF
  http://www.pmwiki.org/wiki/Cookbook/PublishPDF
  http://www.pmwiki.org/wiki/Cookbook/PmWiki2PDF
  http://www.pmwiki.org/wiki/Cookbook/PmGraphViz
  http://www.pmwiki.org/wiki/Cookbook/ASCIIMath
  http://www.pmwiki.org/wiki/Cookbook/MimeTeX
  http://www.pmwiki.org/wiki/Cookbook/PmWikiDraw
  http://www.pmwiki.org/wiki/Cookbook/SourceBlock
  http://www.pmwiki.org/wiki/Cookbook/Comments
  http://www.pmwiki.org/wiki/Cookbook/ICalExport
  http://www.pmwiki.org/wiki/Cookbook/CardsMarkup
  http://www.pmwiki.org/wiki/Cookbook/ChessMarkup


The Network Effect

What's interesting is that once a few converters are created and various base 
type are defined, independent recipes should start to grow more capabilities 
transparently since new export types could potentially become available to 
them.  I envision that the auto conversion properties of this recipe could 
benefit from a "network effect".  Each base type and output type adds to the 
network as a whole and the network of types becomes greater than the sum of 
the whole.  But that is just fantasy for now. :)


I know that I did not list a lot of examples, but my main goal was to make a 
music recipe that was very flexible and easily extensible to handle new input 
and output formats.  I have never liked the alternative solutions to 
embedding images that most people have used:  create an image on disk an 
reference it *2, create a separate script that is completely URL driven *3.  
Neither of these solutions is very extensible and they do not as easily have 
the potential for the network effect mentioned above.  Additionaly, there is 
a lot of shared logic that both of these solutions could benefit from and an 
API makes this possible, as the API grows, any recipes using it could grow 
automatically too.

Thanks for the questions!

-Martin


*1  I implemented this with gnuplot and then realized that gnuplot is not even 
close to being safe to run from webspace.  Primarily it has too many ways to 
load/save to any file on the server to which the web server user has access 
to.  I tried devising ways to strip out any unsafe calls from gnuplot markup, 
but that is too hard to get right since filenames can be created with 
variables and various quoting options.  I am investigating ways to 
potentially jail this process, but it doesn't look easy.

  Better would be to create either an interface that can only execute safe 
stuff or to use another tool that is safer to start with. (I am open to 
suggestions)


*2  A separate image on disk does not have any access control.  It cannot be 
flushed when space constraints are important without URLs failing.  It is 
hard to force these images to be recreated if the converter used to created 
them is enhanced (and the wiki page is not touched.)  Previews can leave 
arround unused previews with no clean way to know when it's safe to clean 
them up.

*3 This is a good solution for some things, particularly if it creates nice 
URLs, but for some data with large sources, the URLs would become unwieldy.  
This solution also does not have any access control and it does not have the 
advantage of being linked to a page so that any cached outputs can be flushed 
when sources are changed.