[pmwiki-users] Re: RE: Fixed: Error in cookbook rename function
jo at durchholz.org
Thu Jun 2 06:45:19 CDT 2005
Patrick R. Michaud wrote:
> This is how my original test suite worked, and it's generally okay
> except I think we sometimes want wildcarding or pattern matching
> capabilities in the expect portion of the test.
Just my thinking.
> And as you mentioned it's really only good for testing markup -- we'd
> also need a testing framework for PmWiki's internal operations such
> as saving files and page history and the like. But perhaps testing
> markup is a good enough start.
I fully agree.
However, for most people working on PmWiki, getting the markups parsed
correctly (and not breaking existing markup) is the most difficult
aspect. So I'd recommend starting with markup testing. Having regression
tests for all the other internals would be nice, too, but since that's
mostly yourself working on that, the value of having regression tests is
One thing that others could start work on are documented interfaces,
such as those listed on PmWiki.Variables and Cookbook.Functions. With
those things documented, people can step in and contribute regression
tests. (It's generally a good idea to have somebody else write them. In
addition to writing one's own tests, of course ;-) )
To integrate such tests, we could set up a new directory pub/test, with
an index.php that serves as a framework to start all the tests, log
results, and list those tests that didn't work out OK. Test suites
should go into subdirectories. (Markup tests should be integrated with
that test script, so that there's one central point where to start the
tests and collect results.)
Since there are *many* things that need testing, it's important to
condense test results. I.e. tests that pass shouldn't be shown at all.
Also, since regression testing shouldn't run multiple times in parallel,
it needs some locking of its own. Something along the lines of:
* If no test is currently running, have pub/test/index.php display the
condensed results of the previous test, and a "Start tests" button.
* If a test is currently running, have pub/test/index.php display the
current state of the test. Bonus points for updating the page whenever
another test result becomes available (i.e. use unbuffered output, poll
for additional test result lines from the log file, and whenever a new
line of log output becomes available, send it to the browser).
More information about the pmwiki-users