[osmosis-dev] Semicolon and Backslash problem

[Frederik Ramm]

Hi,

    API 0.5 will store the character sequence "\\" in the node tags 
column table when a tag contains a backslash, and will store "\s" when 
it contains a semicolon. (Also \e for an equals sign; see lib/tags.rb in 
the rails port.)

The current database server, running Osmosis 0.29.5, generates .osc 
files that contain these escape sequences instead of the "real" thing. 
Uploading a node that contains

     <tag k='test_semicolon' v='left;right' />
     <tag k='test_backslash' v='left\right' />

will yield a minutely diff containing

   <create version="0.5" generator="Osmosis 0.29.5">
     <node id="..." timestamp="..." uid="..." user="..." ...>
       <tag k="test_semicolon" v="left\sright"/>
       <tag k="test_backslash" v="left\\right"/>
     </node>
   </create>

When this diff is applied to a MySQL 0.5 database, all is fine because 
the API expects to see the escaped values there.

However, when this diff is applied to planet file, the resulting .osm 
file will also carry the escaped values, which is highly undesirable, 
and incompatible to the planet files published on planet.osm.org (which 
do not have escape sequences).

Is this a known problem? If not, it is now ;-)

It is a temporary issue because API 0.6 does not escape node tags any 
longer. But I have become aware of the problem because it breaks the 
daily OSM extracts that we at Geofabrik provide (they're built by using 
a planet file and updating it with daily diffs using osmosis --ac). It 
should ideally be fixed server-side. At first I thought to simply run 
the daily diffs through a sed command that un-escapes stuff but this is 
hampered by the fact that only node tags are being escaped that way, so 
if a way tag contains a \s then this really is a "\s" and not a ";". - 
Maybe I should write a quick Osmosis plugin "--unescape-change-nodes" or 
so?

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"