[OSM-dev] 120111 Planet Node Order?

andrzej zaborowski balrogg at gmail.com
Sun Jan 15 18:00:12 GMT 2012


On 15 January 2012 11:26, Lynn W. Deffenbaugh (Mr) <ldeffenb at homeside.to> wrote:
> On 1/15/2012 5:09 AM, Simon Poole wrote:
>> To give you current numbers, I just did (10 days ago) an import on a i7
>> 2600, 16GB box that took 30 hours with the initial import phase running at
>>
>> Processing: Node(1322468k 340.2k/s) Way(120291k 35.64k/s) Relation(1243830
>> 58.30/s)  parse time: 28598s
>>
>> Node stats: total(1322468982), max(1576326287) in 3887s
>> Way stats: total(120291564), max(144049709) in 3375s
>> Relation stats: total(1243833), max(1951174) in 21335s
>
> THAT is the kind of data I was looking for and haven't noticed.  So your
> Ways are about 10% of the rate of the Nodes, and Relations are about twice
> as fast as the Ways.  and there are very few Relations by comparison.  There
> is definitely something different in my nodes going at 45K/s and my ways
> going at 0.12k/s.  I've since aborted that run and restarted with some VM
> changes, but it looks like I've got a ways to go yet.

The smaller cache will affect the ways import rate much more than
nodes so this might be the result.

>
>
>>
>> Because the box has limited memory I did an earlier attempt with -C 8000
>> because I knew that it would swap a lot with -C 12000. However that ran at
>> roughly 4k/s ways and after a couple of hours I aborted it. The good news is
>> that you -can- import on a machine with less memory (good luck keeping up
>> with the updates on a VM though), for example lonvia has imported on a
>> machine with 6GB total (but with -C 12000).
>>
>> The error message you got is weird and may point to an issue with the
>> export process, but it just states that the node cache is going to be less
>> efficient space wise, so IMHO you can simply ignore that.

If Lynn is running a 32-bit system or a 32-bit version of osm2pgsql,
it might be impossible to allocate more than there is physical memory
in the system if it's close to 2-3GB.

Cheers



More information about the dev mailing list