[OSM-dev] 120111 Planet Node Order?

Simon Poole simon at poole.ch
Sun Jan 15 10:09:33 GMT 2012


Lynn,

current versions of osm2pgsql can import .pbf, so you are on the right 
track there. Richards text is good, but it was written when planet files 
were a third of the current size and is just a bit dated. If osm2pgsql 
can't hold the nodes in it's cache it will have to retrieve them from 
the database and that is in my and other peoples experience at least  an 
order of magnitude slower.

To give you current numbers, I just did (10 days ago) an import on a i7 
2600, 16GB box that took 30 hours with the initial import phase running at

Processing: Node(1322468k 340.2k/s) Way(120291k 35.64k/s) 
Relation(1243830 58.30/s)  parse time: 28598s

Node stats: total(1322468982), max(1576326287) in 3887s
Way stats: total(120291564), max(144049709) in 3375s
Relation stats: total(1243833), max(1951174) in 21335s

Because the box has limited memory I did an earlier attempt with -C 8000 
because I knew that it would swap a lot with -C 12000. However that ran 
at roughly 4k/s ways and after a couple of hours I aborted it. The good 
news is that you -can- import on a machine with less memory (good luck 
keeping up with the updates on a VM though), for example lonvia has 
imported on a machine with 6GB total (but with -C 12000).

The error message you got is weird and may point to an issue with the 
export process, but it just states that the node cache is going to be 
less efficient space wise, so IMHO you can simply ignore that.

So add some swap and try again :-)

Simon

Am 15.01.2012 02:57, schrieb Lynn W. Deffenbaugh (Mr):
> I'm a rank amateur at this, can you provide a link on how to use (or 
> what to use instead of) osm2pgsql to import from a pbf instead of 
> planet-120111.osm.bz2?  .pbfs are not mentioned at 
> http://weait.com/content/build-your-own-openstreetmap-server which is 
> the best reference I've found on getting a tile server running for the 
> planet.
>
> Lynn (D)
>
> On 1/14/2012 8:06 PM, Simon Poole wrote:
>> Further tip: use the .pbf Files. It won't help a lot with your 
>> current issue, but is quite a bit faster.
>>
>> Simon
>> -- 
>> Sent from my Galaxy Tab with Kaiten Mail. 
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev




More information about the dev mailing list