[OSM-dev] osm2pgsql for 64-bit IDs

[Igor Brejc]

On Tue, May 24, 2011 at 1:23 PM, Frederik Ramm <frederik at remote.org> wrote:

> Hi,
>
>
> On 05/24/11 12:04, Igor Brejc wrote:
>
>> Yes, but is it really? It's a storage format, you need a 3rd party
>> driver to read it
>>
>
> Same for anything that uses protocol buffers, or for shapefiles, isn't it?


Not necessarily the same thing. Writing protobuf reader is still much easier
than implementing something like http://www.sqlite.org/fileformat.html and
http://www.sqlite.org/fileformat2.html


>
>  and it's optimized for querying, not for storing high
>> volume of data in an efficient manner. And it's a database without a
>> standard schema.
>>
>
> I think that's right (apart fro the "optimized for querying" part perhaps
> because I think it is very much in the hands of whoever creates the
> spatiallite file whether it is good for querying or not).


Even without any indices included, the data is still organized for random
access, which is not really necessary for a "transmission" file format.


>
>
>  I see spatialite as a good way for thick clients to store geo data
>> without the need for an extra DB installation, but not as a good way to
>> exchange data files (as opposed to osm.pbf, for example).
>>
>
> Maybe I misunderstood. I thought that you had said "oh, shapefiles don't do
> 64 bit IDs, maybe we should write a replacement for shapefiles then", and I
> pointed out that such replacements IMHO already exist; I didn't claim that
> they were a good OSM data transport format, and there's no reason to invent
> a new OSM data transport format either IMHO.
>

I'm not talking about OSM-specific data transport format, just something
that would:

   - not have an issue with 64bit IDs
   - be contained in a single file
   - be easy to read/write
   - use efficient binary storage tricks (like the ones PBF uses).

>From reading what spatialite's author has to say on the subject, he finds
spatialite format to be useful because of its inherent querying abilities
but not as a transmission file format:
https://groups.google.com/forum/#!topic/spatialite-users/a1dFsdGUA0A

So I presume Geofabrik will switch to providing spatialite extracts instead
of shapefiles once the 64bit switch becomes a reality?

Igor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20110524/6c47d309/attachment.html>