[OSM-dev] osm2pgsql for 64-bit IDs

[Frederik Ramm]

Hi,

Jochen Topf wrote:
> If we use unsigned ints we have some more time. Problematic would only be
> a few cases where negative IDs are currently used (like in JOSM for data
> thats not yet uploaded to the server). But it seems wasteful to me, to go
> to 64bit a year or so earlier than needed to accommodate this case.

I guess that is a decision that any application must make for 
themselves; there may be some that even today don't support negative IDs 
so they could easily use unsigned ints. Both Maperitive and osm2pgsql 
are occasionally used by people who edit data in JOSM and then run in 
through these tools, so one would have to devise something clever to 
make these scenarios still work.

There are a number of optimisations that I have not pursued, and that 
would IMHO make more sense than trying to buy a little time by using 
unsigned integers. One is using 32-bit IDs for relations and ways, and 
only going 64-bit for nodes (given that the highest way ID is about one 
tenth of the highest node ID). Another is very likely optimizing 
osm2pgsql's internal node cache, which already works in a segmented 
fashion, to store the extra bits with segments rather than with node 
IDs, i.e. like in 80286 times you'd internally use multiple blocks of 
32-bit IDs instead of a full 64-bit address space.

But I felt that fiddling with the node cache - which seems to be a very 
elaborate contraption - was beyond my pay grade ;)

Interestingly, at least on a 64 bit machine, PostgreSQL doesn't seem to 
be overly concerned about using bigints; the database size seemed to 
grow by something like 20% only.

But if someone wants to build an uin32_t version of osm2pgsql, that 
should not require much more than fiddling with the settings at the top 
of osmtypes.h, where various things are defined for 32/64 bit mode.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"