[OSM-dev] Planet dump error (was [OSM-talk] Weird objects showing up in Toronto)

[Dave Stubbs]

On 28/11/2007, Jon Burgess <jburgess777 at googlemail.com> wrote:
>
> On Wed, 2007-11-28 at 12:54 +0000, Dave Stubbs wrote:
> > On 28/11/2007, Tom Hughes <tom at compton.nu> wrote:
> > > In message <a4c775140711280218v7c859a2dq5f424cde05be0ea4 at mail.gmail.com>
> > >         Dave Stubbs <osm.list at randomjunk.co.uk> wrote:
> > >
> > > > But the recycling is actually in the planet:
> > > >
> > > >   <node id="59566115" lat="43.6443335" lon="-79.3943744"
> > > > timestamp="2007-09-24T00:11:28Z">
> > > >     <tag k="created_by" v="JOSM" />
> > > >     <tag k="highway" v="traffic_signals" />
> > > >     <tag k="description" v="Pedestrian crossing signals - do not
> > > > control Clarence Square or Wellington Street" />
> > > >     <tag k=" not activated as of 23 Sept 2007" v="yes" />
> > > >     <tag k="glass" v="yes" />
> > > >     <tag k="amenity" v="recycling" />
> > > >     <tag k="scrap_metal" v="yes" />
> > > >     <tag k="tyres" v="yes" />
> > > >     <tag k="glass_bottles" v="yes" />
> > > >   </node>
> > >
> > > Then the planet is wrong.
> > >
> >
> > I think you're right.
> >
> > I guessed that it was more likely that the API was dropping data than
> > the planet dump was adding it, but it turns out that's a bad guess
> > because of a buffer overrun bug in the planet dump prog.
> >
> > I'm now guessing that that node actually had an extra ';' somewhere in
> > the last tag value.
> > If not then I've no idea what the problem was.
> >
> > I've attached a patch for the planet.c file which fixes the bug I
> > identified and also adds unescaping for the new tag escaping scheme.
> > If someone could look over it, add it to svn and deploy it for the
> > next dump that should hopefully fix this from happening again.
>
> Thanks for diagnosing the problem and providing the fix. I've applied
> the changes to SVN and updated the executable used on dev for the weekly
> dump.
>
> I tweaked the unescaping to pass other \ escaped characters through
> unchanged, e.g. \x becomes x instead of \
>

It might actually be best to pass through the entire escape sequence
unchanged. So \x remains \x -- this would make it consistent with the
API's behaviour which only transforms the \s, \e, and \\ escape
sequences.

The point might actually be moot as I can't actually find any
instances of \ in the current planet except \e,\s,\\ from the last
week... and with the new escaping the \x sequence can't occur. So I'm
not particularly concerned :-)

Dave