[OSM-dev] Unexistant wayed segments in planet?

spaetz osm at sspaeth.de
Mon Sep 17 08:39:31 BST 2007


On Sun, Sep 16, 2007 at 07:54:38PM +0100, 80n wrote:
> > Yes, exactly. Since the script dumps nodes, then segments, then ways,
> > any nodes segments created while segments are being dumped will be
> > missing, as will all segments created while ways are being dumped.
> 
> There would be greater referential integrity if the ways were dumped first,
> then segments and then nodes.
> 
> Since some consumers of planet.osm might expect nodes, then segments then
> ways, the dump could easily create three separate files which are then
> stitched together in the expected order.

Currently the dump is piped straight through bzip2, so we never have an uncompressed copy of the planet on our disk. I changed that in preperation for way larger planets (as e.g. with full TIGER data), because we don't want to store 80GB of uncompressed data in temporary files. Do you know whether three individual bbz2 files can easily be concatenated into one? If yes, making that change should be possible. The only thing that comes into my head is to bzcat those files into a new bz2 file, but I don't know how efficient something like this would be.

I know that osmosis dump from current tables won't be any more consistent. But given Brett's long term goals, of creating consistent snapshots, using history tables and change sets, this would then be possible. Of course, we'll need to weigh whether the requirement to have consistent history tables is worth the benefit of having consistent snapshots.

spaetz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070917/5305b853/attachment.pgp>


More information about the dev mailing list