[OSM-dev] api0.6 - only one value per key?

[Tom Hughes]

Shaun McDonald wrote:

> 1. Rails talks UTF8 to the web
> 2. Rails thinks the database is Latin1
> 3. Rails converts the data from UTF8 to Latin1 to talk to the database.
> 4. The database then converts the latin1 to UTF8
> 5. The database tables are UTF8 so there is a silent truncation when  
> the data is stored to varchar 255 fields based on the conversion of  
> the data in steps 3 and 4. With multibyte chars this is a problem as  
> it can be mid way through the character that the truncation happens so  
> you get the UTF8 errors.

Not quite right..

1. Rails talks UTF8 to the web
2. Rails has the database connection configured as Latin1
3. Rails doesn't take account of that and sends UTF8 anyway
4. MySQL receives UTF8 which it thinks is Latin1 and reencodes it

So the final data is double encoded in UTF8 and the characters of the 
inner encoding may indeed get truncated as you say.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/