[OSM-dev] Osmium - "Corruption" when writing PBF

[Jochen Topf]

Hi!

Some things I noticed in the code:

1. You are supposed to call
 Osmium::init();
in main before doing anything else with Osmium.

2. You should not use the Osmium namespace for your code (in this case your
handler). It will not break anything at the moment, but if Osmium ever gets a
handler called "OsmiumHandlerGridWriter", it will clash with yours.

3. There are some members in OsmiumHandlerGridWriter class that are not
initialized in the constructor. You should probably do that just to make sure.  

4. You do not need to declare/define empty functions like before_nodes(), they
are all declared and defined in the parent class.

5. I don't know what QString does. For instance in this line:
 QString myKey = nIt->key();
does that create a copy of the string? Or do you end up with a pointer to a
the key string which might later be changed in ways its not supposed to change?

I am not sure any of those lead to the problem you describe, but you should
fix them and we'll see how it look afterwards.

Jochen

On Wed, Sep 21, 2011 at 03:48:45AM -0400, Pris Matic wrote:
> Date: Wed, 21 Sep 2011 03:48:45 -0400
> From: Pris Matic <prismatic.project at gmail.com>
> To: Jochen Topf <jochen at remote.org>
> Cc: dev at openstreetmap.org
> Subject: Re: [OSM-dev] Osmium - "Corruption" when writing PBF
> 
> Sorry!
> 
> I totally forgot I had already asked this :)
> 
> http://pastebin.com/GjtgqzkP  -  osmium handler header
> http://pastebin.com/75JUT824  -  osmium handler source
> http://pastebin.com/bSpGj0Ez  -  main.cpp
> 
> It should be possible to just run this on any map file as long as you set
> the bounding box correctly in the handler source file. It takes a single
> osm.pbf file, adds tag data, and outputs 4 quads from the original. It seems
> to work find on some smaller data sets. On larger files, I get the corrupted
> output I mentioned previously...
> 
> Regards,
> 
> Pris
> 
> 
> On Wed, Sep 21, 2011 at 2:47 AM, Jochen Topf <jochen at remote.org> wrote:
> 
> > Hi!
> >
> > You already asked this on Sep 1, and I told you then that we might be able
> > to
> > help you if you post your code. We are not magicians you know!
> >
> > Jochen
> >
> > On Wed, Sep 21, 2011 at 12:57:15AM -0400, Pris Matic wrote:
> > > Date: Wed, 21 Sep 2011 00:57:15 -0400
> > > From: Pris Matic <prismatic.project at gmail.com>
> > > To: dev at openstreetmap.org
> > > Subject: [OSM-dev] Osmium - "Corruption" when writing PBF
> > >
> > > Hi all,
> > >
> > > I'm trying to use Osmium to read in OSM data from *.osm.pbf files, add
> > > certain tag data and then write it out again to *.osm.pbf. I'm able to do
> > > this with smaller files, (city level), but when I try to do
> > state/provincial
> > > level I run into 'corruption', where the tag data becomes completely
> > mixed
> > > up and unusable. It's a strange problem -- the tag keys and values are
> > > 'preserved', but they are all mixed, seemingly at random.
> > >
> > > Here is an example:
> > >
> > > way:
> > >   id=7948229
> > >   version=1
> > >   uid=5168
> > >   user=|43.637614,-79.424110|
> > >   changeset=505735
> > >   timestamp=2007-09-23T19:11:20Z
> > >   tags:
> > >     k=|43.208074,-80.002507| v=|Book Road|
> > >     k=|43.402130| v=|43.191320,-79.935827|
> > >     k=|43.398032,-79.793599| v=|43.191382,-79.935896|
> > >     k=|43.398144,-79.792577| v=|43.191420,-79.936025|
> > >     k=|43.398410,-79.791691| v=|43.191414,-79.936282|
> > >     k=|43.670969,-79.295439| v=|-79.936007|
> > >     k=|43.670595,-79.295294| v=|43.191384|
> > >   node_count=4
> > >   nodes:
> > >     ref=59222301
> > >     ref=59222588
> > >     ref=59222589
> > >     ref=59222590
> > >
> > > I'm saving node geometry data for certain ways. Here's an example of what
> > > the key/value pairs look like when they are correct:
> > >
> > > way:
> > >   id=75168646
> > >   version=1
> > >   uid=5640
> > >   user=|MikeyCarter|
> > >   changeset=5631822
> > >   timestamp=2010-08-29T23:21:59Z
> > >   tags:
> > >     k=|my_startaddr| v=|724|
> > >     k=|my_street| v=|City Road|
> > >     k=|0| v=|48.354900,-89.284862|
> > >     k=|1| v=|48.356371,-89.279541|
> > >     k=|2| v=|48.356387,-89.279429|
> > >     k=|my_endaddr| v=|700|
> > >     k=|3| v=|48.356431,-89.277626|
> > >     k=|avg_lon| v=|-89.280365|
> > >     k=|avg_lat| v=|48.356022|
> > >   node_count=4
> > >   nodes:
> > >     ref=886971214
> > >     ref=887071818
> > >     ref=886990640
> > >     ref=887031646
> > >
> > > However, at some point in the process, they will turn 'random' and become
> > > garbage. I've been stuck on this issue for awhile, and I'd appreciate any
> > > advice.
> > >
> > > -Pris
> >
> > > _______________________________________________
> > > dev mailing list
> > > dev at openstreetmap.org
> > > http://lists.openstreetmap.org/listinfo/dev
> >
> >
> > --
> > Jochen Topf  jochen at remote.org  http://www.remote.org/jochen/
> > +49-721-388298
> >
> >

-- 
Jochen Topf  jochen at remote.org  http://www.remote.org/jochen/  +49-721-388298