[OSM-dev] osm2pgsql for 64-bit IDs

Igor Brejc igor.brejc at gmail.com
Tue May 24 10:24:32 BST 2011

On Tue, May 24, 2011 at 10:52 AM, Jochen Topf <jochen at remote.org> wrote:

> One problem with 64bit IDs is simply that they need twice as much space. If
> you
> store a billion node IDs that might be the difference between needing 4GB
> of
> RAM or 8GB. So I think it is worth it trying to live with 32bit IDs as long
> as possible. Hardware is getting cheaper. So preserving 32bit IDs for a
> year
> longer might mean investments can be postponed and/or we can actually do
> things
> we could not do otherwise, because there is no money for more hardware.
> And while we are at that subject: There is another problem here. Most of
> the
> usual GIS software uses 32bit IDs, when using QGIS with Postgres for
> instance
> it would not accept a 64bit Postgres ID column. (This might have been fixed
> in
> the mean time, I haven't checked for a while.) I have talked about this on
> several occasions to the people who work on these projects and they all
> said,
> they'd work on it. But in the meantime there is an awful lot of software
> around that can't handle this case.
Things usually get fixed only when they are broken. So until we actually
move to 64bit and force things to happen I doubt most GIS software will be
ready for it. Postponing this for one year won't make much difference in
that regard.
As for the storage aspect of things, I can agree with you. But I'm not sure
introducing a temporary system will be cheaper than investing in more
storage. Hardware is getting cheaper, but human labor isn't.

Oh, yes, and shapefiles only allow 32bit IDs.
This could be a more serious issue. I guess in the history of GIS there has
never been such a large geo database as OSM is now becoming. Maybe we (as
the OSM community) should take a proactive stance and propose a new version
of shapefile format that could cope with 64 bits. Yes, I know this is
daydreaming, but shapefile format is getting old anyway. Something using
protocol buffers could be a new way to go - easier to write readers and
writers and taking less space, too.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20110524/35484a3f/attachment.html>

More information about the dev mailing list