[OSM-dev] Merging pbf files problem?

[Jon Burgess]

On Mon, 2010-12-20 at 17:39 -0500, Will Wilson wrote:
> On Mon, Dec 20, 2010 at 5:26 PM, Jon Burgess <jburgess777 at gmail.com>
> wrote:
>         On Mon, 2010-12-20 at 17:07 -0500, Will Wilson wrote:
>         > 1) Trying to use osm2pgsql to import the merged .pbf file
>         directly
>         > died with a generic error immediately (as opposed to
>         importing the
>         > merged XML, which dies 10-15 minutes in). Maybe an error in
>         Osmosis'
>         > pbf handling?
>         
>         
>         Is the error message the same?
>         
> 
> 
> Nope, just:
> 
> 
> Reading in file: /home/will/OSM/workdir/USA.osm.pbf
> Unable to open /home/will/OSM/workdir/USA.osm.pbf
> Error occurred, cleaning up

The PBF support was only recently added to osm2pgsql. You need to fetch
the latest sources and verify that it finds the right protobuf-c
libraries during the configure step.

>  
>         > 2) What's the expected behavior for Osmosis when merging
>         contiguous
>         > but non-overlapping extracts? Would I have gotten the same
>         problem if
>         > I'd started with .osm.bz2 ?
>         
>         
>         Since the USA is such a large proportion of the whole planet
>         file
>         (probably over 50%) I believe most people import the whole
>         planet file
>         instead of trying to extract the US data.
> On Mon, Dec 20, 2010 at 5:20 PM, Frederik
> Ramm <frederik at remote.org> wrote:
>         If Jon's suspicion is true (likely), download the full planet
>         file instead and run osm2pgsql with a bounding box option to
>         restrict it to the US.
>  
> Yeah, my initial plan was to do what the two of you suggest, and just
> use planet.osm with a bounding box; but there's next-to-zero
> documentation that I could find on keeping a subset of planet.osm
> (with bbox or bpolygon) updated with daily or hourly diffs. Any
> reading suggestions?

I'm not sure if it is documented and the process is a compromise. The
simplest thing is to apply the world diffs and put up with the small
amount of extra data which appears for the rest of the world. It should
be possible to apply a bbox filter to the diffs too but there are some
edge cases which can result in missing data. In real-life these edge
cases will probably be so rare that they won't cause any real issues. 

> 
> Also, does .pbf format support changesets yet?

No idea. Since most changesets are small the time and space savings of
PBF are generally less important.

   Jon