[pmwiki-devel] database tools and hacks in PmWiki

[marc]

Crisses said...
> On Dec 10, 2006, at 1:13 PM, marc wrote:
> 
> > Crisses said...
> >
> >> I have thought through the product/service end of the shopping cart,
> >> but not the order storage end.  Storing credit card/bank information
> >> in flat files is absolutely out of the question for me
> >
> > This isn't necessary. All the payment gateways I've used don't require
> > it and it's better to steer clear of the legal issues. Repeat payment
> > mechanisms are almost always available too. Even if you wanted to
> > collect CC info to pass through on a secure connection, there's no  
> > need
> > to store it.
> 
> If you've ever programmed an API for a payment system, you're ahead  
> of me.

Not in these days of PHP and the web ;-)

> I was only READING the documentation for the QuickBooks payment  
> gateway the other day, and YES, one would be dealing directly with  
> the payer's credit card info on the PHP/pre-API end.  We're talking  
> 100% must have SSL, etc.

Yes, if you deal directly with the gateway you are in heavier territory, 
but I made the presumption - not unreasonable I thought in Pmwiki-land - 
that folk would be using payment providers such as WorldPay et al. Here 
in the UK, I prefer protx to WorldPay.

(WorldPay are a merchant account + payment provider + gateway [as are 
PayPal, in effect, although I have a big problem with them not being a 
bank, and certainly not behaving like one], whereas protx are a 
payment provider and gateway, so you need a merchant account - usually 
via your bank, to use their service.)

I suspect that QuickBooks is a gateway (or a poor payment provider 
service, there are (or were) quite a few of those) - I must check out 
the terminology for the two layers, but I think that gateway and payment 
provider are correct.

> Must store info, except the handy-dandy  
> code on the back of the card that we shouldn't even be requesting the  
> user to enter, and absolutely MUST NOT store.

I just had a quick squint at the protx docs and they say (their caps):

"YOU DO NOT NEED TO COLLECT CREDIT OR DEBIT CARD DETAILS. All your site 
needs to do is calculate the total cost of the order in whatever 
currency your site operates and present the user with a confirmation 
page, summarising their order. On this page will be a Proceed or 
Continue button which, when clicked, will initiate the payment process 
outlined in the following sections."

So they are not forbidding you collecting the card info. (protx have an 
excellent system these days, and a scan through their guide 
(VSPServerProtocolandIntegrationGuideline.pdf) is quite informative.) A 
quick glance at the WorldPay docs shows that there is no provision for 
providing card info, so it's not an option there - I'm sure that they 
have an account type that does permit this, though.
 
> >> one reason I
> >> love PayPal from a liability/programming point of view -- absolutely
> >> no need to store financial information on the system.
> >
> > Well, I loath PayPal - and enjoy a steady stream of work moving folk
> > away from them :-) - but collecting payment info prior to  
> > commencing the
> > payment transaction is not a requirement with any of the payment
> > gateways I've used.
> 
> If you have the experience, again, you are far far ahead of me.  For  
> me, it's all theory.  I'm reading up on things.  If your payment  
> processors are storing customer's info so you can request returns or  
> issue credits, etc. then it's better than what I've seen & read so far.

I have an active WorldPay account here - that's not a recommendation ;-)
To manage transactions, you go here:

  http://support.worldpay.com/admin/

login, enter a transaction code, and, in amongst all the transaction 
information, it provides a "Refund details" section where there's a 
button for "Make full refund", and a "Refund amount" input field with an 
associated "Make partial refund" button.

Just FYI, the only card detail shown is the card type (i.e. Visa, Amex).

> If you have experience with shopping carts and payment processors,  
> could you please give comments and contributions to the ideas for the  
> shopping cart for PmWiki, or write one?

I have a cart that I could probably package into a recipe, and on which 
we could build, since it is obviously geared to my needs. It is based 
on sessions, since I don't want a user to have to be logged in to add 
items to a cart, nor make a purchase. This model might not work for 
everyone.

Another issue is where to grab the product info from. It's no problem 
expanding this in the class and recipe, but some folk will want to 
provide it via markup args:

  (:AddProduct prodcode=PRODCD price=12.34 desc='Prod desc' label='Buy 
me':) 

others via page text variables or some other PmWiki page munging and 
others from a database. I don't have a problem with any of these, but 
it'll need folk to get involved to help write the bits that they need.

Another issue is multiple currencies. I do this for GDP, EUR and USD, 
but I grab my fex rates from the payment provider. Again, this might not 
suit everyone.

Payment processing is more tricky because each payment provider has a 
different interface and procedure for payment processing. PayPal is 
extraordinarily weak here because it doesn't provide an automatic 
callback. This is usually the deal breaker for small businesses once 
they understand how much extra work this creates for them.

The other point to note is that each business will have different 
requirements on conclusion of a successful transaction (or a failed 
one). That is, different processing in response to the bank's callback.

We could build a generic payment processing recipe for PmWiki, but it 
would be a lot of work. I also suspect that many folk would be interest 
in an interface to PayPal, and that is not something that I'd be 
interested in. Folk who need that should, imo, build it specifically for 
PayPal, since it will require all sorts of functionality to plug the 
holes in PayPal's process that don't exist elsewhere.

I have a payment process working for two payment processors, but, as I 
mentioned, I doubt that these would be useful to others, other than as 
skeletons. For example, I store order details in a db until the callback 
confirms the sale, then update the db. The requirements of each users' 
db will be different. That said, we could discuss and define generic 
handling for each payment provider and develop a class to do this, 
including provision for data being stored in PmWiki or in a db.

> I am struggling with the  
> amount of complexity behind it.  If you have dealt with the payment  
> processor APIs and would like to see a shopping cart system in  
> PmWiki, can you help?

It's not simple mainly because each payment processor does things 
differently. If there were a standard - there should be, but, as with 
all cartels they work hard to ensure that there isn't - then the problem 
would be very straightforward.
 
> >> The moment the
> >> shopping cart system requires working with other merchant processing
> >> systems that require the shopping cart to handle sensitive financial
> >> information, we're entering very mucky territory in the way PmWiki
> >> handles data.  You must store the customer's credit card in case of a
> >> product return, or needing to issue a credit, etc.
> >
> > This isn't how it works, in my experience. The payment gateway  
> > provides
> > an interface that allows you to access the transaction and either  
> > refund
> > or make a part repayment. At no point do you need the credit card
> > information.
> 
> That's not what I've seen.  I would welcome a list of payment  
> gateways, but note that some customers come with an API/system they  
> want to use because it's tied in with their in-store terminal  
> already, or their bank, or some other deal they've already signed on  
> to....

Maybe things are different in the US. My experience with US banks has 
been pretty tortuous, so I don't doubt it ;-) Over here, the payment 
provider market has had a shakedown. Most of the payment providers ended 
up in the hands of the banks and thus normal service has been resumed 
;-)

> The rest I do encourage to use PayPal, at $0 up-front costs, and most  
> of my customers are probably under $1000/month sales.

Disregarding my strong ethical objections to PayPal, using them for 
business, however small, is false economy. The absence of an automatic 
callback means that accounts processing cannot be automated. This 
creates unnecessary work (=cost), which is in no way outweighed by the 
cash savings.

> >> But you can't store it in a text file.
> >
> > Why not? Technically, I mean, since I don't this either.
> 
> Technically? Of course you can.  Why not?  Because most people are on  
> shared hosted servers.  That means they can get the www process to  
> peek at your file system, maybe.  Possibly.
> 
> >> Do we use GPG?  That doesn't work, because the web server would need
> >> access to the keys.  It's a pain, and so far a database is the best
> >> answer I have.
> >
> > You could store the key outside the web space and include the file.
> 
> See about shared hosting situations.  I don't trust flat file storage  
> on multiuser systems.
> 
> Heck, nowadays you can't trust your hosting system's employees not to  
> steal customer info and sell it -- even banks and financial  
> institutions can't trust their employees.  In a court if someone said  
> "How did you protect your customers' information" I want to be able  
> to honestly say I used all reasonable means.  Flat files are not all  
> reasonable means.  I have yet to have a hosting situation where I  
> can't get access to a MySQL database if I want or need one.  It's not  
> the most secure thing ever, but it's one of the reasonable and  
> securer means.

If someone has the nous to find your encryption key and unravel you 
decryption process - heck I can hardly understand mine and I wrote it! - 
then finding the db config info and accessing the db is unlikely to be 
beyond them, I suspect. But I obviously agree with your sentiment. The 
only time I'd consider storing sensitive info is with a machine on my 
own turf. That said, with two servers to a 1U rack space these days, you 
could walk out with one under your hat ;-)

> The safest is to abstain entirely from storing any  
> damming information.

As I said, that's why I don't do it.

-- 
Best,
Marc