[OSM-dev] Bin-test failed (different test)

Scott Crosby scrosby at cs.rice.edu
Sun Oct 3 16:56:42 BST 2010


On Thu, Sep 30, 2010 at 7:41 PM, Brett Henderson <brett at bretth.com> wrote:

> On Fri, Oct 1, 2010 at 12:23 AM, Curt Nowak <nowak at bwl.uni-hildesheim.de>wrote:
>
>> Hello dev,
>>
>> I also tested osmosis 0.37-SNAPSHOT with the bin-support for various
>> countries downloaded at Geofabrik.
>> In contrast to the -xml tasks I *sometimes* ran into an exception using
>> the -bin tasks in combination with the --used-node task.
>>
>>
>>
>> What's up with that? Why is there suddenly a negative user id involved
>> that is not present in the .osm file of Slovenia? Btw: This does *not*
>> happen for all tested input files. E.g. Andorra worked fine.
>>
>
> For users that aren't public, a user called NONE is used within the Osmosis
> pipeline.  It has an id of -1.  But that is typically not written to data
> files.  The bin tasks may need to update their handling of private users ...
>
>
I think I know the cause:

When metadata is stripped during writing. No UID is stored. If that file is
later read the binary reader has to plug in some UID for downstream osmosis
and OsmUser.NONE is used. If the output of this processing is written to a
binary file, without metadata stripping, the code will preserve the UID=-1
to the file. Now, when that file is read again, the reader passes the UID=-1
to downstream osmosis, which doesn't like it.

The root cause is osmosis assumes and requires that all entities have
metadata, and I offer a feature that allows it to be stripped for a15% gain
in filesize and processing speed. Brett, how do you suggest I handle the
case of stripped metadata to downstream osmosis?


FIXES:
   1. I suspect that somewhere in Frederik's pipeline is an
omitmetadata=true, but downstream processing reads a file written with that
flag set, and writes a file without that flag.
   2. Error out when writing a binary file that contains UID<0, to catch
mistakes of type 1.Those situations are sorta silly as space is wasted
storing useless metadata.
   3. Transparently map UID<0 in the file to OsmUser.NONE when reading.
   4. Make osmosis not care if UID's are negative.

My preference is fix #4, but warn if negative UID's are being written, to
detect use errors with of the discussed in fixes #2 and #1.

Scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20101003/4c610429/attachment.html>


More information about the dev mailing list