Failing to send a file in iso-8859-1
Hello everyone, I would like to send the CSV content of a scalar variable as a file to the browser by using send_file in Dancer1. My sample code looks like this: send_file( \$str, content_type => 'text/csv', filename => 'foo.csv', ); The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that. My first attempt was to just encode the content (which I should do anyway, I guess): $str = Encode::encode('iso-8859-1', $str); send_file( \$str, content_type => 'text/csv', filename => 'foo.csv', ); However, it seems like the content gets somehow magically re-encoded to UTF-8. Moreover, the browser receives a content type header of Content-Type:text/csv; charset=utf-8 So I have tried the following variants: $str = Encode::encode('iso-8859-1', $str); send_file( \$str, content_type => 'text/csv', charset => 'iso-8859-1', filename => 'foo.csv', ); ---- $str = Encode::encode('iso-8859-1', $str); send_file( \$str, content_type => 'text/csv', encoding => 'iso-8859-1', filename => 'foo.csv', ); ---- $str = Encode::encode('iso-8859-1', $str); send_file( \$str, content_type => 'text/csv; charset=iso-8859-1', filename => 'foo.csv', ); The first two do not seem to change anything, the last one results in a content type header of Content-Type:text/csv; charset=iso-8859-1; charset=utf-8 In all cases, the content of the file is UTF-8 encoded. Does anyone know how I can fix this issue? Thanks a lot and best wishes, Lutz
Hi! 2016-10-05 19:59 GMT+03:00 Lutz Gehlen <lrg_ml@gmx.net>:
send_file( \$str, content_type => 'text/csv', filename => 'foo.csv', );
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that.
As far as I understand, the charset is set in config, so you could try to override the setting in your route like: set 'charset' => 'iso-8859-1; Never tried, but it should work. -- Wbr Kõike hääd, Gunnar
Hi Gunnar, thank you for your reply. On Wednesday 05.10.2016 20:58:22 WK wrote:
2016-10-05 19:59 GMT+03:00 Lutz Gehlen <lrg_ml@gmx.net>:
send_file(
\$str, content_type => 'text/csv', filename => 'foo.csv',
);
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that.
As far as I understand, the charset is set in config, so you could try to override the setting in your route like:
set 'charset' => 'iso-8859-1;
Never tried, but it should work.
Yes. this works. However, it changes charset permanently (for the current process) and I don't get the chance to change it back because send_file does not return. Is there no way to set it temporarily just for this send_file? Thanks and best wishes, Lutz
On 10/06/2016 02:56 PM, Lutz Gehlen wrote:
Hi Gunnar,
thank you for your reply.
On Wednesday 05.10.2016 20:58:22 WK wrote:
2016-10-05 19:59 GMT+03:00 Lutz Gehlen <lrg_ml@gmx.net>:
send_file(
\$str, content_type => 'text/csv', filename => 'foo.csv',
);
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that.
As far as I understand, the charset is set in config, so you could try to override the setting in your route like:
set 'charset' => 'iso-8859-1;
Never tried, but it should work.
Yes. this works. However, it changes charset permanently (for the current process) and I don't get the chance to change it back because send_file does not return. Is there no way to set it temporarily just for this send_file?
Hello Lutz, you could (re)set the charset in the "before" hook to the normal one. Regards Racke -- Ecommerce and Linux consulting + Perl and web application programming. Debian and Sympa administration.
On 06/10/16 15:03, Stefan Hornburg (Racke) wrote:
On 10/06/2016 02:56 PM, Lutz Gehlen wrote:
Hi Gunnar,
thank you for your reply.
On Wednesday 05.10.2016 20:58:22 WK wrote:
2016-10-05 19:59 GMT+03:00 Lutz Gehlen <lrg_ml@gmx.net>:
send_file(
\$str, content_type => 'text/csv', filename => 'foo.csv',
);
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that. As far as I understand, the charset is set in config, so you could try to override the setting in your route like:
set 'charset' => 'iso-8859-1;
Never tried, but it should work. Yes. this works. However, it changes charset permanently (for the current process) and I don't get the chance to change it back because send_file does not return. Is there no way to set it temporarily just for this send_file? Hello Lutz,
you could (re)set the charset in the "before" hook to the normal one.
Regards Racke
There ought to be a way to set charset as an option to send_file. I've created an issue: https://github.com/PerlDancer/Dancer2/issues/1260 Lutz: could you please check this issue and add further comment if needed? Also let us know if you'd like to have a go at a PR. If not I'll create a patch. R. PeteM
Hi Pete, On Thursday 06.10.2016 15:32:05 Peter Mottram wrote:
On 06/10/16 15:03, Stefan Hornburg (Racke) wrote:
On 10/06/2016 02:56 PM, Lutz Gehlen wrote:
On Wednesday 05.10.2016 20:58:22 WK wrote:
2016-10-05 19:59 GMT+03:00 Lutz Gehlen <lrg_ml@gmx.net>:
send_file(
\$str, content_type => 'text/csv', filename => 'foo.csv',
);
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that.
As far as I understand, the charset is set in config, so you could try to override the setting in your route like:
set 'charset' => 'iso-8859-1;
Never tried, but it should work.
Yes. this works. However, it changes charset permanently (for the current process) and I don't get the chance to change it back because send_file does not return. Is there no way to set it temporarily just for this send_file?
you could (re)set the charset in the "before" hook to the normal one.
There ought to be a way to set charset as an option to send_file.
I've created an issue: https://github.com/PerlDancer/Dancer2/issues/1260
You have created the issue in the Dancer2 project, is this intended?
Lutz: could you please check this issue and add further comment if needed? Also let us know if you'd like to have a go at a PR. If not I'll create a patch.
Thank you for encouraging me to implement this myself. I'd like to give it a try. I might need some assistance with the pull request, though. I am familiar with git and have an account on github, but I have never submitted a pull request before. I guess that I will fork https://github.com/PerlDancer/Dancer to my github account, then do my work, and then somehow submit a pull request. Is this the correct procedure? How should I go about it with respect to Dancer1/Dancer2? I myself need the feature in Dancer1. Should I implement it there? Or both? Or is Dancer1 even somewhat feature-frozen? Cheers, Lutz
On 10/07/2016 08:47 AM, Lutz Gehlen wrote:
Hi Pete,
On Thursday 06.10.2016 15:32:05 Peter Mottram wrote:
On 06/10/16 15:03, Stefan Hornburg (Racke) wrote:
On 10/06/2016 02:56 PM, Lutz Gehlen wrote:
On Wednesday 05.10.2016 20:58:22 WK wrote:
2016-10-05 19:59 GMT+03:00 Lutz Gehlen <lrg_ml@gmx.net>:
send_file(
\$str, content_type => 'text/csv', filename => 'foo.csv',
);
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that.
As far as I understand, the charset is set in config, so you could try to override the setting in your route like:
set 'charset' => 'iso-8859-1;
Never tried, but it should work.
Yes. this works. However, it changes charset permanently (for the current process) and I don't get the chance to change it back because send_file does not return. Is there no way to set it temporarily just for this send_file?
you could (re)set the charset in the "before" hook to the normal one.
There ought to be a way to set charset as an option to send_file.
I've created an issue: https://github.com/PerlDancer/Dancer2/issues/1260
You have created the issue in the Dancer2 project, is this intended?
Lutz: could you please check this issue and add further comment if needed? Also let us know if you'd like to have a go at a PR. If not I'll create a patch.
Thank you for encouraging me to implement this myself. I'd like to give it a try. I might need some assistance with the pull request, though. I am familiar with git and have an account on github, but I have never submitted a pull request before.
I guess that I will fork https://github.com/PerlDancer/Dancer to my github account, then do my work, and then somehow submit a pull request. Is this the correct procedure?
You are doing your work in a branch on your fork, e.g. pr/send-file-charset and push that branch to your fork. After that PR will be offered to create on the Github website.
How should I go about it with respect to Dancer1/Dancer2? I myself need the feature in Dancer1. Should I implement it there? Or both? Or is Dancer1 even somewhat feature-frozen?
Dancer1 is somewhat feature frozen, but this would be bugfix IMHO. At any rate, David Precious as Dancer 1 maintainer will determine whether to apply it. Regards Racke -- Ecommerce and Linux consulting + Perl and web application programming. Debian and Sympa administration.
Hi Racke, On Thursday 06.10.2016 15:03:42 Stefan Hornburg (Racke) wrote:
On 10/06/2016 02:56 PM, Lutz Gehlen wrote:
On Wednesday 05.10.2016 20:58:22 WK wrote:
2016-10-05 19:59 GMT+03:00 Lutz Gehlen <lrg_ml@gmx.net>:
send_file(
\$str, content_type => 'text/csv', filename => 'foo.csv',
);
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that.
As far as I understand, the charset is set in config, so you could try to override the setting in your route like:
set 'charset' => 'iso-8859-1;
Never tried, but it should work.
Yes. this works. However, it changes charset permanently (for the current process) and I don't get the chance to change it back because send_file does not return. Is there no way to set it temporarily just for this send_file?
you could (re)set the charset in the "before" hook to the normal one.
You are right, of course. This is a feasible workaround as a last resort. Cheers, Lutz
On 6/10/2016 11:56 PM, Lutz Gehlen wrote:
send_file(
\$str, content_type => 'text/csv', filename => 'foo.csv',
);
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that. As far as I understand, the charset is set in config, so you could try to override the setting in your route like:
set 'charset' => 'iso-8859-1;
Never tried, but it should work. Yes. this works. However, it changes charset permanently (for the current process) and I don't get the chance to change it back because send_file does not return. Is there no way to set it temporarily just for this send_file?
There should be no need to change your apps charset. If you look at any of the HTTP specs, a Content-Type header looks like: Content-Type: text/html; charset=utf-8 Without digging into the Dancer1 code, send_file(\$str, content_type => 'text/html; charset=utf-8', filename => 'foo.html' ); should DWIM (it will in Dancer2). Substitute your own mime types and encoding as needed. Hope that helps, Russell @veryrusty
Hi Russell, On Friday 07.10.2016 12:27:26 Russell Jenkins wrote:
On 6/10/2016 11:56 PM, Lutz Gehlen wrote:
send_file(
\$str, content_type => 'text/csv', filename => 'foo.csv',
);
The problem is that external requirements force me to send the file in iso-8859-1 and it seems that I cannot manage to achieve that.
As far as I understand, the charset is set in config, so you could try to override the setting in your route like:
set 'charset' => 'iso-8859-1;
Never tried, but it should work.
Yes. this works. However, it changes charset permanently (for the current process) and I don't get the chance to change it back because send_file does not return. Is there no way to set it temporarily just for this send_file?
There should be no need to change your apps charset.
If you look at any of the HTTP specs, a Content-Type header looks like:
Content-Type: text/html; charset=utf-8
Without digging into the Dancer1 code,
send_file(\$str, content_type => 'text/html; charset=utf-8', filename => 'foo.html' );
should DWIM (it will in Dancer2). Substitute your own mime types and encoding as needed.
Thank you for your suggestion. However, this does not work in Dancer1. I have already tried this approach (see my first email) and the default character set is still attached to the Content-Type header resulting in Content-Type: text/csv; charset=iso-8859-1; charset=utf-8 Moreover, the data get still encoded in utf-8. I believe that the relevant code is in Dancer::Handler::render_response: ... my $charset = setting('charset'); my $ctype = $response->header('Content-Type'); if ( $charset && $ctype && _is_text($ctype) ) { $content = Encode::encode( $charset, $content ) unless $response->_already_encoded; $response->header( 'Content-Type' => "$ctype; charset=$charset" ) if $ctype !~ /$charset/; } ... Best wishes, Lutz
participants (5)
-
Lutz Gehlen -
Peter Mottram -
Russell Jenkins -
Stefan Hornburg (Racke) -
WK