[osmosis-dev] Semicolon and Backslash problem

[Brett Henderson]

On Tue, Feb 3, 2009 at 12:03 PM, Frederik Ramm <frederik at remote.org> wrote:

> Hi,
>
>    API 0.5 will store the character sequence "\\" in the node tags
> column table when a tag contains a backslash, and will store "\s" when
> it contains a semicolon. (Also \e for an equals sign; see lib/tags.rb in
> the rails port.)


Okay, osmosis isn't doing any of that so will need to be fixed.


>
>
> The current database server, running Osmosis 0.29.5, generates .osc
> files that contain these escape sequences instead of the "real" thing.
> Uploading a node that contains
>
>     <tag k='test_semicolon' v='left;right' />
>     <tag k='test_backslash' v='left\right' />
>
> will yield a minutely diff containing
>
>   <create version="0.5" generator="Osmosis 0.29.5">
>     <node id="..." timestamp="..." uid="..." user="..." ...>
>       <tag k="test_semicolon" v="left\sright"/>
>       <tag k="test_backslash" v="left\\right"/>
>     </node>
>   </create>
>
> When this diff is applied to a MySQL 0.5 database, all is fine because
> the API expects to see the escaped values there.
>
> However, when this diff is applied to planet file, the resulting .osm
> file will also carry the escaped values, which is highly undesirable,
> and incompatible to the planet files published on planet.osm.org (which
> do not have escape sequences).
>
> Is this a known problem? If not, it is now ;-)


I remember thinking something was probably broken when that escaping was
introduced but forgot to check it out.  I'm surprised nobody has noticed it
until now :-)


>
>
> It is a temporary issue because API 0.6 does not escape node tags any
> longer. But I have become aware of the problem because it breaks the
> daily OSM extracts that we at Geofabrik provide (they're built by using
> a planet file and updating it with daily diffs using osmosis --ac). It
> should ideally be fixed server-side. At first I thought to simply run
> the daily diffs through a sed command that un-escapes stuff but this is
> hampered by the fact that only node tags are being escaped that way, so
> if a way tag contains a \s then this really is a "\s" and not a ";". -
> Maybe I should write a quick Osmosis plugin "--unescape-change-nodes" or
> so?


It sounds like a bug in the mysql tasks so it really should be fixed there.
Do you have time to write the escape and unescape string routine?  If not
I'll take a look as soon as I can.

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20090203/492a45c8/attachment.html>