[cgiapp] Streaming Status Updates for a Long-running Run Mode

[eric.berg at barclayscapital.com]

> -----Original Message-----
> From: cgiapp-bounces at lists.openlib.org 
> [mailto:cgiapp-bounces at lists.openlib.org] On Behalf Of Michael Peters
> Sent: Wednesday, October 07, 2009 11:44 AM
> To: CGI Application
> Subject: Re: [cgiapp] Streaming Status Updates for a 
> Long-running Run Mode
> 
> On 10/07/2009 11:35 AM, eric.berg at barclayscapital.com wrote:
> 

> > The approach I'm working on now is to have a global status  hash for
each
> > file being processed, which I'll update as I run each subprocess and
> > when each completes.  Then I've thrown in a quick run mode  that
simply
> > sends that hash back as JSON and I'm putting some logic  into my
page to
> > periodically make a request for this runmode so I can  update the
page
> > with current status info.
> 
> I really wouldn't do that. Don't tied up your web server for long 
> running tasks just so that you can wait to show a status to the user. 
> Even something as simple as Unix "at" for a simple queue 
> would be better.

I wouldn't do that either, but I've had no end of issues with running
subprocesses from Apache2.  Among them is that of the environment's not
being passed to the subprocess, the way apache messes with STDOUT, and
the fact that it insists on starting CGI's (mod_perl handlers) in the
directory in which they are invoked, regardless of the current working
directory of the mod_perl handler.

I just ran across the 'at' solution, which I like quite a bit, however
it turns out that we do not receive the email for the users that the web
server runs under and 'at' sends error and other confirmation
information via email, so I have no idea at this point why my
subprocesses aren't running.  Yes.  It's heinous to tie up apache with
running subprocesses, but regardless of whether I'm using
Apache2::SubProcess->spawn_proc_prog() or IPC::Run3, I still get screwed
one way or another.

I wrote a wrapper for running via a number of different methods when
this problem first emerged after migrating to apache2.  I used
Module::Pluggable, so all I had to do was to implement a runner plugin
that uses 'at', which was nice, but it's failing and I'll have to wait
until I get our IT crew to understand and help with the email problem.
(don't ask)

> 
> > On a related note, is there a way to have cgiapp send content back
to
> > the client during the processing of a run mode instead of at the end
> > when the rm returns?
> 
> No, not for Ajax. And it's not a limitation of cgiapp, but of
HTTP/Ajax. 
> For non-Ajax you use Non-Parsed-Headers (NPH) which means you  tell
C::A 
> to not send headers and you then instead print them  yourselves. And
then 
> periodically print more things to the client (like some JS  that
updates 
> a progress bar, etc). But like I said in the other email, this won't 
> work for Ajax.

Understood that you can't do this with Ajax, but how would you do it
with cgiapp in a non-ajax context?

> 
> -- 
> Michael Peters
> Plus Three, LP
_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________