[cgiapp] Bug-Problem with CGI.pm, fcgid, CGI::Application and utf-8?

[Giannis Economou]

Hello,

I'm writing to report a nasty situation/bug I'm facing. It is quite
strange to me, I can not find out which component might be fully
responsible (maybe just CGI.pm or the combination of C::A and CGI.pm)
and I gave some hours to isolate the situation as much as possible and
reproduce it. Maybe someone is also interest or can help.
I explain and afterwards I send you some sample code you can try.

I have some applications written, now in production installation, using
C::A. They are different apps, using many CGI::App modules and C::A
Plugins and they show output (html) in utf-8. Among other they also use
TT2, but this is also irrelevant to the problem.
All started when I accidentally found out that when cookies are disabled
(in firefox or ie) then web pages do not display the utf-8 correctly,
but when cookies are enabled, everything is displayed correctly!


I've managed to isolate the problem (as you can guess I started being
suspicious with many modules in the way before stripping everything in a
new test case).
It happens only with:
- C::A applications (simple cgi script are not having the problem)
- running on my server (Linux, Apache/2.0.52, Perl v5.8.5), I haven't
tried other servers, but I guess it will be also have the problem
- only when run under fcgid acceleration (plain cgi execution has
consistent behavior), I also tried the latest fcgid module on my Apache
no luck
- only when using CGI.pm (with CGI::Simple this problem is not
happening, utf-8 is always displayed right) - my CGI.pm is version 3.31.

It is frustrating, mostly because I was really happy with fcgid as
acceleration of my apps until today. Now I had to switch to plain cgi again.



Here is a sample source that demonstrates and reproduce the problem,
using the bare/minimal modules to reproduce it:

############### (instance.fcgi) ###############

#!/usr/bin/perl
use CGI::Fast;
use WebApp;

while (new CGI::Fast) {
my $self = WebApp->new();
$self->run();
}


############### (WebApp.pm) ###############

package WebApp;
use base 'CGI::Application';

sub setup {
my $self = shift;
$self->start_mode('test');
$self->mode_param('rm');
$self->run_modes(
'test' => 'show_test',
);
$self->header_add( -charset => 'utf-8' );
}

sub show_test {
my $self = shift;
my $cookie = $self->query->cookie(-name=>'test', -value=>'foo');
$self->header_add(-cookie=>$cookie); #set the cookie

return "Hello ... Καλημέρα!"; #this is 'Good morning' in Greek, file is
saved as utf-8
}

1;


Now:
- when you run the instance.fcgi as fcgid script, cookie is set and in
browser you get garbage in the Greek text (encoding is correctly set in
browser).
- when you run the instance.fcgi as plain cgi script, everything is fine
- when you comment the cookie setting line in WebApp.pm, everything is
working fine, regardless of fcgid or plain cgi.
- when you use CGI::Simple instead of CGI.pm (for example use a
CGI::Simple object instead of $self->query which is a CGI.pm in the
example above to create the cookie), everything is working fine (fcgi /
plain cgi, set cookie / do not set cookie, everything is fine).

Here is the output of CGI::Application, as sent in browser, but as
unicode codes:
Correct:
Hello ... \xce\x9a\xce\xb1\xce\xbb\xce\xb7\xce\xbc\xce\xad\xcf\x81\xce\xb1!

Erroneous:
Hello ...
\xc3\x8e\xc2\x9a\xc3\x8e\xc2\xb1\xc3\x8e\xc2\xbb\xc3\x8e\xc2\xb7\xc3\x8e\xc2\xbc\xc3\x8e\xc2\xad\xc3\x8f\xc2\x81\xc3\x8e\xc2\xb1!


Best regards,
Giannis Economou