[cgiapp] handling data and control flow better (longish)

[David N. Blank-Edelman]

Hi-

So these general "how to think about stuff when you are first  
starting" questions on the list and all of the really helpful, gentle  
responses has finally pushed me to get out of lurk mode to finally  
write my own similar message. A number of months ago I finished my  
first medium-sized CGI::App-based webapp. In the process, I ran into a  
few things that seemed harder than they should be re: CGI::App, but I  
plowed through them. Now that  the project is largely complete, I'd  
like to bring back some of those experiences to this list to find out  
what I should do in the future when faced with similar issues. I can't  
tell if the stuff I ran into was due to my lack of experience or  
approach vs. something in CGI::App, but I'm almost certain it was the  
former. Anything you can suggest that can help me improve how I'm  
working/thinking about these things would be much appreciated.

Here's a brief description of the webapp: needed something to assist  
our users request that their mailboxes be migrated from one system to  
another. I started by writing code that crawled our filesystem trying  
to make intelligent guesses about where the user's stored their mail  
and stored these guesses in a database. The webapp I wrote let user's  
log into it, it retrieved the best guess about their mail storage  
location from the database and also confirmed some personal info with  
the user. If the guess was wrong according to the user, it would take  
in the new info and then attempt to log into our IMAP server to check  
if the user's input was indeed correct (it would show the user the  
folders it found using their mail root path and check if that was  
correct). After the last screen, the webapp actually uses SOAP to  
create the user on the new mail system, writes out the shell script  
necessary to migrate the user and sends mail to our trouble ticket  
system to log the request.  All of this was done with field validation  
(C::A::P::ValidateRM), sessions, and  decent logging  
(C::A::P::LogDispatch). So, not a huge application (~750 lines of code  
w/9 run modes), but still respectable.

While writing this app, I ran into the following pitfalls:

1) At times it was hard to keep all of the pieces of the program's  
state in my head at the same time. For each run mode, I was juggling  
three sets of parameters:
       C::A::P::Session's parameters ($self->session->param())
       CGI's query parameters ($self->query->param(), mostly fed in  
through variables previous run modes definedin forms)
       the data I got out of my pre-loaded database

(didn't use CGI::App's param(), that would have made four)

2) Keeping those things straight was harder because their values  
depended on which run mode I was in at the time and how I got there.  
That was made more complex by C::A::P::ValidateRM, which basically can  
toss you back to the run mode you thought you just left if something  
doesn't pass validation. This meant that every run mode had to  
understand whether it was being entered "fresh" (and hence things like  
the CGI query parameters were more meaningful than the session data)  
or being entered because it was being held back in class due to an  
error (and hence the session data might be more relevant than the CGI  
query data). On top of this, I was also using C::A::P::Forward when I  
needed it. I used it to pass control over to (non-/)recoverable error  
run modes which added another tricky bit of juggling.

So at every moment, I had to be cognizant of:
     1) where I was (run mode)
     2) what data I was handed (CGI query)
     3) what other data did I know about/should I save  
(C::A::P::Session and database fields)
     4) which was the "most current" parameter amongst them
     5) how exactly did I get into a run mode (control flow) or at a  
particular place in a run mode
     6) and where did I need to go next
     7) other stuff needed to program the task at hand

I'm not the sharpest tack in the carpet, but I think I'd argue that  
that's a whole lot of stuff for anyone to keep track of simultaneously  
as they are trying to write a program

In retrospect, I think the added complexity of having to manually cope  
with multiple passes through the same run mode due to validation  
errors was a particularly unpleasant part of the programming process.  
I'm hoping this isn't coming off as whiny or as a rant, but  I  
remember thinking "Why am I having to work so hard? There must be a  
better way."  I believe I had brief thoughts about how something  
transactional would be nice. For example, if validation failed, the  
program would roll back its state so that the run mode could treat  
things largely as if it were entered anew vs. having to have basically  
define two sub-run modes (one for first entry, the other for  
validation errors).

I developed some basic coping tools to deal with these things (happy  
to talk about them), but I'd really like to hear if other people have  
had the same experience as I did. Do you find it hard to keep all of  
the data and control flow stuff straight in your head when developing  
using CGI::App? If so, how do you deal with it?

Thanks for any advice you can offer (and for listening to my questions).

      -- dNb

P.S. I should also mention that mostly, the above issues  
nonewithstanding, it was a blast developing with CGI::Application. I  
really appreciated all of the hard work that went into making it as  
cool as it is and I definitely plan to use it/Titanium in my next  
project.