Localization Project Management Log

A few weeks ago I posted on an I18n problem with IBM Websphere that was causing corrupted characters to display. In short, Websphere had told the browser to ignore the stated page encoding (UTF-8) and to display the page as if encoded for Latin-1. Not the Jedi way.

Our engineers had to get this escalated to tier 3 with IBM support. This seemed ridiculous to me, because we can’t have been the only Websphere site trying to display Spanish and Portuguese, and other people must have complained about such a silly problem, but it took tier 3 to get us a solution, and that’s all that matters.

The short answer: we need to change all of our top level jsp’s to explicitly set the response encoding to UTF-8 (response.setContentType(“text/html; charset=UTF-8″);) . Once the engineers had done that, the container finally returned a consistent result in UTF-8. It’s still a bit confusing why the UTF-8 encoding was returned on some pages and not on others but it all seems to work now, so we happily closed this case with IBM.

I append the entire response from IBM, simply so that it will live in one more place on the Web for future searches.

Two points before diving into your server1-non-working trace:

I. How WebSphere application server set Default Response Encoding:

If autoResponseEncoding is true:
1. Check request locale, set it if exists
2. Get encoding from request.getCharacterEncoding()
3. set the encoding according to the above locale
4. set to default ISO-8859-1

finally setContentType() with the charset set to above encoding.

Please note that autoResponseEncoding is independent from
autoRequestEncoding or client.encoding.override.

II. From Servlet Spec:
1. setContentType(): only work if response has not been committed (i.e
before getWriter)
2. Dispatch Include (SRV.8.3)
Any attempt to set headers or call any method that affects the
headers of the response will be ignored.

==============================================

Now this is your server1-non-working’s analysis:

1. autoResponseEnding set the encoding according to step (I.3) above:
from the locale (en_us)
thus the encoding is ISO-8859-1

setContentType type –> text/html; charset=ISO-8859-1

2. The request to /home.wfl will result in a dispatch forward to
[/WEB-INF/jsp/home.jsp] which in turn including other resources:

[/WEB-INF/jsp/home.jsp]

setContentType type –> text/html

+include /WEB-INF/jsp/include/header.jsp
++including /WEB-INF/jsp/include/includes.jsp]
++including /WEB-INF/jsp/include/syncstatus.jsp]

(there are several attempt to call the setContentType() within the
including JSP but got ignored …though I do not see any attemp to set
charset to UTF-8)

3. getWriter() with the encoding that found/set in step 1: ISO-8859-1.

=================================

You might want to check the top level JSP (i.e. home.jsp in this case)
and setContentType accordingly and before the response is committed.
Do not set it in the including resources as it will be ignored.
————————

Thank you for using IBM products and support.