Lost in translation - encoding

I am currently trying to proces开发者_开发技巧s a http post request from a J2me application to a Websphere 6 server which communicates with other systems via MQ messaging. The Queue character set is Cp037 and the message retrieved from the queue is Cp1256.

Everything works fine up to the point where I try to send a response in Arabic to the j2me client as such:

String respStr = new String(orginalMsg, "Cp1256");
response.getOutputStream().write(bytes);

The reponse from the servlet is set with:

• Character Encoding: UTF-8
• Content Type: text/plain;charset=UTF-8

I subsequently read the response in J2me with:

inputStreamReader = new InputStreamReader(connFact.getInputConnection(),"UTF-8");  
buffer = new StringBuffer((int) length); // If no length, default to 256 buffer
char[] data = new char[128];
int total = 0x00;
int read = -1;
do
{
    read = inputStreamReader.read(data);
    if (read > 0x00)
    {
        total += read;
        buffer.append(data, 0x00, read);
    }
} while (read != -1 && !cancel);

I am running the code in the WTK emulator which I know supports UTF-8 arabic characters as I can display harcoded strings on a form.

However when it comes to display the result from the response only ? characters are displayed.

I have tried to do the conversion programmatically instead of letting Websphere do the implicit conversion but I get the same result.

Any pointers would be much appreciated.

From your comment, you imply that you are doing this:

//these bytes are Cp037 encoded
bytes = txtMsg.getText().getBytes("Cp037");
//you don't seem to be using respStr
String respStr = new String(orginalMsg, "Cp1256");
//you've told this response it is UTF-8, but the bytes are Cp037
response.getOutputStream().write(bytes);

If you get character data (string/char array) from the queue (txtMsg.getText()), set the response content type and use a writer to write the response data.

response.setContentType("text/plain;charset=UTF-8");
response.getWriter(txtMsg.getText());

The writer will transcode the UTF-16-encoded character data to UTF-8.