[cgiapp] Data validation of file uploads

[Nicholas Bamber]

Comments below in no particular order

* Once processing is finished the data is presumably written to a file 
or a database. and the memory can be reclaimed.

* One of my concerns about Data::FormValidator::Constraints::Upload and 
Data::FormValidator::Filters::Image is that as far as I can see they 
must be loading the entire file into memory at least temporarily. And 
yet the user still has no access to the data or has not actually even 
got a non -temporary file name.

* These sort of issues are the same that any successful website must 
address. There are books on scaling websites (such as splitting your 
server into a lightweight front-end caching proxying websever and a 
modperl heavy back-end server.)

* One can use rate limiting, authentication and other techniques to 
deter DOS attacks. In fact in my use cases upload will only be possible 
for the beneficial owner of the website anyway.

* I've actually got my code working (though I am just about to try it 
out in anger) so you can see more concretely what I am thinking of at 
http://github.com/periapt/CGI-Application-Plugin-AJAXUpload. You'll 
notice that this module uses CGI::Upload for the heavy lifting, 
Data::FormValidator for the data validation, and 
CGI::Application::Plugin::JSON for communicating back to the client.

Michael Peters wrote:
> On 09/20/2010 03:01 AM, Nicholas Bamber wrote:
>
>> For the purposes of my discussion we can assume that, say 5M, is a
>> reasonable maximum file size. The modules that I am comparing with do
>> slurp the entire file into memory. And by only providing a file handle
>> they force the caller to do it more than once.
>
> But even 5M is still a lot to have your processes grow by on each 
> request. For instance, a common mod_perl setup with say 100 apache 
> children could grow by half a gig really fast and result in total 
> machine lockup. It's not an edge case to not want to expose yourself 
> to a DOS attack.
>