[cgiapp] CGI::Application status update from the maintainer

[Mark Stosberg]

Bill--

> I've sort of taken my own path to create web apps, and while it works for me, I've gotten some blowback for it a few times over the years, but I really don't mind that, so I'll explain it.
>
> I have a small demo "Note Pad" app at www.raspberryperl.com:
>
> 	app   ->  http://raspberryperl.com/cgi-bin/NoteApp/notes.cgi
> 	code -> http://raspberryperl.com/NoteAppCode/

I stated to look at the code, but I didn't get very far until I ran into
this notice:

# Do not read, copy, distribute, execute, run, or use
# this code without express written permission from
# William H. Stephenson.

At that point, I was compelled to immediately close the file and stop 
reading.

> When a user creates a note I save the data in a file:
>
> 	open(my $NOTE, "> $note_path/$notepad_number")
> 	or &Note_home("Error 3: That didn't work");
> 	$cgi->save($NOTE);
> 	close $NOTE;

I see. I hope that $note_path and $notepad_number are handled
appropriately, so that a user can't end up overwritting unintended file
son our file system.

> When they want to view or edit a note I load the data into a CGI object:
>
> 	$NOTE = new CGI($FILE);  # Throw out the old $NOTE, replace it with a new one
> 	close $FILE;
> 	return ($NOTE);

Using a representation of an HTTP response as a data storage and
retrieval system makes me shudder. Someone here has already suggested
using XML instead. A format like XML or JSON would offer some greater
flexibility.

I would also suggest considering a DBI backend, such as DBD::CSV,
DBD::AnyData or DBD::SQLite2. This way, your code is a good position to
"scale up" to using a full SQL database if that's ever desired, and you
have a standard set up tools and techniques to use between large
projects that use SQL and your smaller projects too.

I've done a lot of small projects with CGI::Application, and I set a
fairly low bar before I start to involve a SQL database.

Even on low-cost shared hosting accounts, SQL databases are readily
available.

>
> And use HTML:Template to display it:
>
> 	$template->param(note_subject => $cgi->param('note_subject'));

HTML::Template has a feature called 'associate' which makes this kind of
pass-through easier:

     https://metacpan.org/module/HTML::Template#Miscellaneous-Options

> I generally use the epoch time of creation to name the file. This gives me an easy way to search by date for documents.
>
> 	1344045036.txt
>
> Or perhaps assign a document number:
>
> 	0001.txt

Either scheme might work well, until you have a lot of documents and
need to search by some other key, requiring opening *every* file to
check something.

> In that case, if you know the document number, finding and loading the
> data is incredibly fast because you're using the system's b-tree to
> locate it. The trick (if it can be called that) is creating a unique
> directory for each user so the app knows where to find their
> documents. Put each different document type in a unique sub-directory
> and you've really narrowed down where to find what you're looking for.
>

> Now, my logic for this being practical is that for the most part
> database engines were designed to solve just a few problems:
>
> 1. Slow processors took a long time ripping through a bunch of files
> while searching for a specific piece of data.

> 2. RAM was expensive and a lack of enough RAM made managing large
> numbers of files impractical and limited what an OS could support.

> 3. Hard Drive space was expensive, so storing a bunch of files was
> also expensive.
>
> But now, RAM and ROM is comparatively cheap, and processors and OSs
> are way faster than they were 15 years ago, so this approach keeps
> becoming more practical for some applications. In my case, the users
> of the apps I build will never create hundreds of thousands of files.
> Ten thousand would be the upper ends of what they'd ever create, so
> this actually works pretty good, and it's very simple to build and
> maintain.

I thought the beauty of SQL was that you expressed *what you wanted*,
not *how to make it happen*.

I much prefer that to code that open and closes files, scans content
line by line, and greps for values.

But I agree with the overall sentiment that the best tools for complex
projects are often not the same as the best tools for small projects.

> I use a flexible interpretation of a MVC design approach in my apps,
> and it would seem to me that if there were ever a necessity to move to
> a SQL database it should be fairly easy to change the routines to get
> and put data, and to load the existing data into the database. If
> that's the case, then my approach might be useful for RAD (rapid
> application development) if nothing else.

It sounds like you've thought about how you might scale up if you needed
to.

> For my own sake, what I am trying to do is limit complexity. Not at
> all costs, but I do set a high bar for the returns. Complexity, in my
> opinion and experience, gets expensive fast. I think it's fair to
> point out that a SQL database engine may offer another point of entry
> for hackers to create mayhem. I'm not saying that my approach is more
> secure, but it seems to me that you must constantly be staying on top
> of issues specific to any database engine you're using, and that may
> create additional expense which may be unnecessary.

I also pursue simplicity as a goal. Sometimes it be a challenge to
recognize which of two options is simplist.

> I understand that if you must have a Relational database my approach
> may not work, but again, with the apps I've created I've not come
> across that need. So I believe there are some good reasons to have a
> simple way, like that in CGI.pm and CGI:Session to store and load data
> using files.

I think you'll find that there will continue to be easy-to-use tools
available to store and access data in files from Perl.

     Mark