[OSM-dev] Planet dump error (was [OSM-talk] Weird objects showing up in Toronto)

[Jon Burgess]

On Wed, 2007-11-28 at 12:54 +0000, Dave Stubbs wrote:
> On 28/11/2007, Tom Hughes <tom at compton.nu> wrote:
> > In message <a4c775140711280218v7c859a2dq5f424cde05be0ea4 at mail.gmail.com>
> >         Dave Stubbs <osm.list at randomjunk.co.uk> wrote:
> >
> > > But the recycling is actually in the planet:
> > >
> > >   <node id="59566115" lat="43.6443335" lon="-79.3943744"
> > > timestamp="2007-09-24T00:11:28Z">
> > >     <tag k="created_by" v="JOSM" />
> > >     <tag k="highway" v="traffic_signals" />
> > >     <tag k="description" v="Pedestrian crossing signals - do not
> > > control Clarence Square or Wellington Street" />
> > >     <tag k=" not activated as of 23 Sept 2007" v="yes" />
> > >     <tag k="glass" v="yes" />
> > >     <tag k="amenity" v="recycling" />
> > >     <tag k="scrap_metal" v="yes" />
> > >     <tag k="tyres" v="yes" />
> > >     <tag k="glass_bottles" v="yes" />
> > >   </node>
> >
> > Then the planet is wrong.
> >
> 
> I think you're right.
> 
> I guessed that it was more likely that the API was dropping data than
> the planet dump was adding it, but it turns out that's a bad guess
> because of a buffer overrun bug in the planet dump prog.
> 
> I'm now guessing that that node actually had an extra ';' somewhere in
> the last tag value.
> If not then I've no idea what the problem was.
> 
> I've attached a patch for the planet.c file which fixes the bug I
> identified and also adds unescaping for the new tag escaping scheme.
> If someone could look over it, add it to svn and deploy it for the
> next dump that should hopefully fix this from happening again.

Thanks for diagnosing the problem and providing the fix. I've applied
the changes to SVN and updated the executable used on dev for the weekly
dump.

I tweaked the unescaping to pass other \ escaped characters through
unchanged, e.g. \x becomes x instead of \

	Jon