[OSM-dev] osm2pgsql for 64-bit IDs
frederik at remote.org
Sat May 21 17:43:44 BST 2011
Jochen Topf wrote:
> If we use unsigned ints we have some more time. Problematic would only be
> a few cases where negative IDs are currently used (like in JOSM for data
> thats not yet uploaded to the server). But it seems wasteful to me, to go
> to 64bit a year or so earlier than needed to accommodate this case.
I guess that is a decision that any application must make for
themselves; there may be some that even today don't support negative IDs
so they could easily use unsigned ints. Both Maperitive and osm2pgsql
are occasionally used by people who edit data in JOSM and then run in
through these tools, so one would have to devise something clever to
make these scenarios still work.
There are a number of optimisations that I have not pursued, and that
would IMHO make more sense than trying to buy a little time by using
unsigned integers. One is using 32-bit IDs for relations and ways, and
only going 64-bit for nodes (given that the highest way ID is about one
tenth of the highest node ID). Another is very likely optimizing
osm2pgsql's internal node cache, which already works in a segmented
fashion, to store the extra bits with segments rather than with node
IDs, i.e. like in 80286 times you'd internally use multiple blocks of
32-bit IDs instead of a full 64-bit address space.
But I felt that fiddling with the node cache - which seems to be a very
elaborate contraption - was beyond my pay grade ;)
Interestingly, at least on a 64 bit machine, PostgreSQL doesn't seem to
be overly concerned about using bigints; the database size seemed to
grow by something like 20% only.
But if someone wants to build an uin32_t version of osm2pgsql, that
should not require much more than fiddling with the settings at the top
of osmtypes.h, where various things are defined for 32/64 bit mode.
Frederik Ramm ## eMail frederik at remote.org ## N49°00'09" E008°23'33"
More information about the dev