[OSM-dev] pbf2osm development has started [code to test!]

[Stefan de Konink]

On Thu, 30 Sep 2010, Frederik Ramm wrote:

> Stefan de Konink wrote:
>>  When this tool is polished enough (must include some bbox'ing then) we
>>  can think about osm2pbf.
>
> Speaking of "polished": The program currently produces invalid XML because " 
> and & are not escaped, leading to lines like

Yes, Roeland pointed that out as well yesterday. We have discussed an 
escape table. Maybe first parsing the entire string table, alternatively 
doing it for each instance.


> Other than that, it runs about twice as fast as Osmosis so that's a good 
> sign. What does the README mean when it says: "At the time of writing it can 
> only handle dense nodes"?

...that the README is already outdated ;)

But what I already discussed with Scott, we *need* a good 'has everything' 
PBF file. Something that can test a parser and has expected output.

For example Osmosis only generates dense nodes, not the 'other notation'.

Also Osmosis currently implements deflate, not bzip2/lzma. (At the time of 
writing I implemented bzip2 as well, but couldn't test it.)


There are a few things on the todo before going for the osm2pbf myself:
- xml escaping
- mmap input
- lzma
- allow the use of the bbox
  + based on the index
  + based on geos like solution

Roeland wanted to go for a library so other tools could call getNextNode() 
etc. without fiddling with the internal structures.

My personal interest is going for output that support output that can be 
used to 'copy into' data into a database. And extend the current protocol 
for diff support. (Hence: replacing osmsucker-ykw)


Stefan