[cgiapp] CGI::Application status update from the maintainer

[Bill Stephenson]

> There not yet specific plans for how CGI.pm will be replaced. How do you
> use the save/restore feature? 

Thank you, Mark. I appreciate your taking my concerns under consideration.

I've sort of taken my own path to create web apps, and while it works for me, I've gotten some blowback for it a few times over the years, but I really don't mind that, so I'll explain it.

I have a small demo "Note Pad" app at www.raspberryperl.com: 

	app   ->  http://raspberryperl.com/cgi-bin/NoteApp/notes.cgi
	code -> http://raspberryperl.com/NoteAppCode/

When a user creates a note I save the data in a file:

	open(my $NOTE, "> $note_path/$notepad_number")  
	or &Note_home("Error 3: That didn't work");
	$cgi->save($NOTE);
	close $NOTE;

When they want to view or edit a note I load the data into a CGI object:

	$NOTE = new CGI($FILE);  # Throw out the old $NOTE, replace it with a new one
	close $FILE;
	return ($NOTE);

And use HTML:Template to display it:

	$template->param(note_subject => $cgi->param('note_subject'));

I generally use the epoch time of creation to name the file. This gives me an easy way to search by date for documents.

	1344045036.txt

Or perhaps assign a document number:

	0001.txt

In that case, if you know the document number, finding and loading the data is incredibly fast because you're using the system's b-tree to locate it. The trick (if it can be called that) is creating a unique directory for each user so the app knows where to find their documents. Put each different document type in a unique sub-directory and you've really narrowed down where to find what you're looking for.

Now, my logic for this being practical is that for the most part database engines were designed to solve just a few problems:

1. Slow processors took a long time ripping through a bunch of files while searching for a specific piece of data.
2. RAM was expensive and a lack of enough RAM made managing large numbers of files impractical and limited what an OS could support.
3. Hard Drive space was expensive, so storing a bunch of files was also expensive.

But now, RAM and ROM is comparatively cheap, and processors and OSs are way faster than they were 15 years ago, so this approach keeps becoming more practical for some applications. In my case, the users of the apps I build will never create hundreds of thousands of files. Ten thousand would be the upper ends of what they'd ever create, so this actually works pretty good, and it's very simple to build and maintain.

I've done tests on ripping through 5000 files (with an app like the demo linked above) to find a unique string in one or more files and the result times are not that bad. I suspect they'd be faster if I used ModPerl or FastCGI, but I deploy on a VPS so that's not been an option. My tests resulted in the initial search being a bit slow, but a second search is pretty fast, so I suspect Apache is cacheing some data but I really don't know much about how that works. 

I have some apps with several thousand users, and some with over 100,000 files in the system. They all run on a VPS and they're pretty darn fast, so again, I'm not finding a compelling reason to move to a SQL database engine. 

I use a flexible interpretation of a MVC design approach in my apps, and it would seem to me that if there were ever a necessity to move to a SQL database it should be fairly easy to change the routines to get and put data, and to load the existing data into the database. If that's the case, then my approach might be useful for RAD (rapid application development) if nothing else.

For my own sake, what I am trying to do is limit complexity. Not at all costs, but I do set a high bar for the returns. Complexity, in my opinion and experience, gets expensive fast. I think it's fair to point out that a SQL database engine may offer another point of entry for hackers to create mayhem. I'm not saying that my approach is more secure, but it seems to me that you must constantly be staying on top of issues specific to any database engine you're using, and that may create additional expense which may be unnecessary.

I understand that if you must have a Relational database my approach may not work, but again, with the apps I've created I've not come across that need. So I believe there are some good reasons to have a simple way, like that in CGI.pm and CGI:Session to store and load data using files.

Kindest Regards,

Bill Stephenson