[cgiapp] Data validation of file uploads

[Nicholas Bamber]

I think I have been completely misunderstood.

Please explain how you can dynamically resize an image without looking 
at the entire data in memory.

By "Data validation" I assumed that everyone would be using the 
"Data::FormValidator" and related modules.
This class has the concept of "Filters" (cleaning up the data) and 
"Constraints" (rejecting or accepting the data).
The upload modules in this hierarchy (very image oriented) from my 
reading of their code must be reading in the entire
file into memory. If you have an issue with that please take it up with 
the authors of those modules. My issue was that the
way these modules have done it is ghastly and that I was refusing to use 
those modules.

In principle I can accept that reading an entire file into memory does 
pose some risks. But the approach to dealing with those risks depends on 
the circumstances and it does not relate to the question I was asking.

cgiapp-request at lists.openlib.org wrote:
> Send cgiapp mailing list submissions to
> 	cgiapp at lists.openlib.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://lists.openlib.org/mailman/listinfo/cgiapp
> or, via email, send a message with subject or body 'help' to
> 	cgiapp-request at lists.openlib.org
>
> You can reach the person managing the list at
> 	cgiapp-owner at lists.openlib.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of cgiapp digest..."
>
>
> Today's Topics:
>
>    1. Re: 	 (Mark Fuller)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 20 Sep 2010 15:11:04 -0700
> From: Mark Fuller <azfuller at gmail.com>
> Subject: Re: [cgiapp] Data validation of file uploads
> To: CGI Application <cgiapp at lists.openlib.org>
> Message-ID:
> 	<AANLkTikn_G6ihXuOk+a852=1EzH284iG8J0Lc3DTzJWA at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> On Mon, Sep 20, 2010 at 7:08 AM, Nicholas Bamber <nicholas at periapt.co.uk> wrote:
>   
>> * Once processing is finished the data is presumably written to a file or a
>> database. and the memory can be reclaimed.
>>     
>
> Can I ask why you don't want to write the incoming stream to a temp
> file the "normal" way?
>
> The only reason I can think of would be security concerns. That it's
> highly sensitive data. In that case, it seems like it would be better
> to transmit a shared secret to the client through an https connection,
> and encrypt the data as it's sent (through javascript?). Or, modify a
> copy of cgi.pm or cgi::simple to encrypt the chunked incoming data as
> it's written to the temp file.
>
> >From something you said earlier, it sounded like you just want to
> untaint the data before it's written to disk. If that's all it is, I
> don't understand how untainted data written to a temp directory would
> be risky.
>
> Mark
>
>
> ------------------------------
>
> _______________________________________________
> cgiapp mailing list
> cgiapp at lists.openlib.org
> http://lists.openlib.org/mailman/listinfo/cgiapp
>
>
> End of cgiapp Digest, Vol 36, Issue 8
> *************************************
>