Android, UTF8 - How Do I Ensure UTF8 Is Used For A Shared Preference

How do I ensure UTF8 is used for a shared preference menu? I have an android preferences menu that allows a user to set their name, amongst other things. I need to know how to conv

Solution 1:

The key is to understand the difference between UTF-8 and Unicode.

• Java processes characters and strings in memory using Unicode. Each character is stored in two bytes.
• When text is transmitted between processes (eg to a web server) or it is written to/read from disk, the internal representation is converted into an over-the-wire format. This is the encoding or decoding. UTF-8 is the most popular, but other formats include:
  • UTF-16
  • ISO 8859-1

In your question, you mention that the XML files are encoded in utf-8: That is good, and you will be able to put foreign characters in the files, but that specifies the encoding only for that specific XML file.

These XML files will be compiled into Android resources and will contain the correct values (you can check it if you like in the debugger, or by preserving the intermediate Java resource files from the build chain).

The problem is almost certainly where you send data to and receive data from the HTTP server, specifically where that data is converted between the bytes on the network and a Java String. Currently you are not setting it in the request - this can be done as described in the documentation for Apache HTTPClient.

Although the server might already require/assume this, it's certainly a good thing to state clearly in the request.

You also need to ensure that the server (the one in Rails 3 - How to handle PG Error incomplete multibyte character):

• Is expecting UTF-8
• Decodes the request using a UTF-8 decoder
• Encodes the response using UTF-8 encoding

(Sorry, but I don't know Ruby on Rails so I don't know how to specifically help there).

Back in the Android end, you also need to ensure that your HTTP library is decoding the response with the UTF-8 decoder. If you handle this yourself, ensure that the String constructor you use is this one, and the argument is "utf-8":

• public String (byte[] data, String charsetName)

Once BOTH the client and the server are using UTF-8, your problems will be resolved.

To help debugging here, I suggest:

1. A number of logging statements on server and client that print the relevant strings as close as possible to the HTTP code

2. Running with the client configured to talk through a debugging proxy. Examine the request and response and check that they are indeed UTF-8. Proxies include:

   • Charles
   • WebScarab
   • Fiddler