[OSM-dev] Storing node locations on ways
Jochen Topf
jochen at remote.org
Wed Apr 20 17:32:59 UTC 2016
On Mi, Apr 20, 2016 at 06:59:35 +0200, Christoph Hormann wrote:
> On Wednesday 20 April 2016, Jochen Topf wrote:
> >
> > Adding node locations to ways is a pain (and needs more than 32 GB
> > RAM these days for a full planet if you want to get it done in
> > reasonable time). So I experimented with adding the node locations to
> > the ways once and storing it in the OSM files. Works for XML, PBF,
> > and OPL. See my blog post at
> > https://blog.jochentopf.com/2016-04-20-node-locations-on-ways.html
> > for the full story.
>
> From perspective of database design this of course introduces redundancy
> in the data which in turn bears the risk of introducing
> inconsistencies. For storing data temporarily during processing this
> is not generally a big deal because the processes writing and reading
> this data are well defined but as a general data exchange and storage
> format this would be highly problematic - depending on how a program
> reads the file it will potentially get different results if there are
> inner inconsistencies in the file.
Well, there are always ways of screwing up the data...
> On a more general note: for the purpose of storing the OSM planet data
> in a form that is efficient to use and update wouldn't it make sense to
> store the node locations in a flat file separately from the rest
> instead of storing everything in a dynamically structured format like
> XML or PBF that can essentially only be read sequentially?
Sure. That would also make sense for some application. But doesn't solve
the problem I am trying to solve here, that is allowing you to work with
OSM data without much RAM.
> Another thing: you say you throw away untagged nodes that are not member
> of any way - don't forget nodes can also be relation members. I don't
> know any practical case where an untagged node is a relation member but
> in principle this is possible and could make sense.
Thats true. I thought about adding the node location also to the relation
member, but, as you say, in most cases you'll need the tags of the member,
too, which would make this a whole lot more complicated. So I can't help
with that use case currently.
Jochen
--
Jochen Topf jochen at remote.org http://www.jochentopf.com/ +49-351-31778688
More information about the dev
mailing list