[OSM-dev] Binary OSM; the first pass encoder

[Stefan de Konink]

Marcus Wolschon wrote:
> So you are trying to bend a perfectly good communication-format into
> an inperfect, mutable on-disk-format
> at the expense of more space and more indirection.

1) There is something needed for first download. I think for this a 
binary protocol is required. You proposed a protocol with embedded 
strings. We try to (over) optimise it using an unique string table.

2) It could be possible that we make an on disk OSM cache.

2a) That for initial pull takes a smart approach, but for the cache an 
indexed one.

2b) And for diffs a binary diff on cache files on disk.


> What kind of diffs where intended? I reat it as a
> generation of a complete new binary-file, then do a binary diff of the
> two where large chunks will stay
> the same and compress that diff and have the client rebuild it's
> complete index after applying it.

In an optimal fashion; a compressed delta.

> What are the actual requirements and the metric that is to be optimized?
> Maybe you two are taking off in different directions.

My intention was only to reduce the filesize of the planet. In any case 
I'll export it to CSV and import it to SQL. Anything beyond that is 
something I would love to help with, including server -> client 
operation, but is not something I would use myself as 'user'.


Stefan