[OSM-dev] Osmosis bug when using 'completeWays' option?

[Karl Newman]

On Dec 2, 2007 2:44 PM, Frederik Ramm <frederik at remote.org> wrote:
> Hi,
>
> > > > If you're working on a
> > > > particular area area, you might want to start with a simple bounding
> > > > box first (no completeWays) to limit the data that is buffered.
>
> > > Currently I'm working on a small area, but it will target the entire world.
> > > Maybe I need to split the world into a few sub areas first (e.g. Europe,
> > > North America, South America, Afrika and Asia) before cutting them into
> > > small pieces.
> >
> > I should talk to Brett about making some sort of shared entity
> > cache/buffer that could be re-used by multiple downstream tasks, so in
> > the case of a "tee" with "completeWays" bounding boxes there would be
> > only one copy of all the entities instead of one set per tee.
>
> Maybe the pre-cutting of a bounding box could be automated. Based on
> the assumption that no way is longer than "x", the area filter could
> hold only references to those objects which are *not* inside the
> selected area but in the vicinity ("x" around the bounding box), for
> possible later inclusion.
>
That's a possibility; now solve for "x" :-)

> As far as I understand, you're keeping the full objects. My Perl
> polygon extractor only kept the object IDs (in a bit set) and
> afterwards either used seek() to jump back to the beginning of the
> relevant section in the input file to retrieve the missing objects (or
> simply re-opened it if it was not seekable). Also, for effective
> output ordering, it created three temporary output files (for nodes,
> ways, relations) and concatenated them at the end of the operation. I
> fear it will be very hard to put such optimisations into Osmosis as
> they don't fit well with the pipeline concept.
>
Well, it does keep the object IDs in a bit set, but the pipline idea
is the problem--the source of the entity data could be a database. The
pipeline paradigm doesn't lend itself to seeking backwards. That's why
we store the entities (three separate temporary files, here, too).

Karl