<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 1/14/2012 7:56 PM, Simon Poole wrote:
<blockquote
cite="mid:7eea2130-477b-4e4b-916d-21affd231c41@email.android.com"
type="cite">Believe me, if you can't allocate enough memory (you
should be running a 64bit OS and have at least enough swap
allocated if you don't have sufficient memory) for the cache for
the import in question, it is going to be very very very very
slow.<br>
</blockquote>
<br>
With System Monitor reporting only 53% memory in use, I don't think
memory is the reason for the slow Way import. The 1.329 billion
nodes imported reasonably consistently and memory use hasn't changed
with the transition from Nodes to Ways. I really think it has more
to do with the "out of order node" error from the planet file, but
google hasn't helped me find any reference to what that really
means.<br>
<br>
<blockquote
cite="mid:7eea2130-477b-4e4b-916d-21affd231c41@email.android.com"
type="cite">
<br>
Don't be confused by the Florida or whatever import running fast,
that is peanuts in comparison to importing a full planet (1.4
billion nodes or so).<br>
</blockquote>
<br>
Understood. I did the Florida import just to make sure the
tool-chain worked. Didn't want to run the whole planet just to test
it.<br>
<br>
Lynn (D)<br>
<br>
<blockquote
cite="mid:7eea2130-477b-4e4b-916d-21affd231c41@email.android.com"
type="cite">
<br>
Simon <br>
<br>
<div class="gmail_quote"><br>
<br>
"Lynn W. Deffenbaugh (Mr)" <a class="moz-txt-link-rfc2396E" href="mailto:ldeffenb@homeside.to"><ldeffenb@homeside.to></a> schrieb:
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
<pre style="white-space: pre-wrap; word-wrap:break-word; font-family: sans-serif">Actually, the instructions I was following called for -C 2048
(<a moz-do-not-send="true" href="http://weait.com/content/build-your-own-openstreetmap-server">http://weait.com/content/build-your-own-openstreetmap-server</a>), but when
I did that I got the following error (running in a 4GB Ubuntu 10.04 VM):
> Out of memory for dense node cache, reduce --cache size
I had to reduce to -C 1800 to make it even run. Nodes imported nice and
fast (43.6k/s), but I received the following error:
> Out of order node 244067335 (33792779,7) - this will impact the cache
> efficiency
which to my novice reading implies that there was an issue in the Planet
file dealing with node ordering. And the Way import is running MUCH
slower than I would have thought given what I've read. 0.12k/s is WAY
less than 43.6k/s. I know ways are slower, but I
figured maybe 10% of
the speed worst case, not this much slower.
Lynn (D)
PS. I haven't seen a -C 12000 recommended anywhere. And interestingly
with -C 2048 complaining , -C 1800 is still only using 2GB of the 3.7GB
available according to Ubuntu's System Monitor.
On 1/14/2012 6:10 PM, Simon Poole wrote:
>
> You do have -C 12000 set?
>
> Haven't seen the error message before, but if you set the node cache
> smaller than necessary to store -all- nodes importing will be very
> slow (naturally if you don't have enough memory the machine will swap,
> but it sill still be faster).
>
> Simon
>
> Am 14.01.2012 23:16, schrieb Lynn W. Deffenbaugh (Mr):
>> Greetings,
>>
>> I've been lurking in the group for a long time now and finally got
>> motivated to try putting together my own Tile server. I go!
t
>> everything working, the whole way through mod-tile using a Florida
>> extract as my data to keep things moving along. It was good.
>>
>> Then I downloaded planet-120111.osm.bz2 and started importing it
>> yesterday. It completed the nodes and is now working on the Ways,
>> but I received on error message:
>>
>> > Out of order node 244067335 (33792779,7) - this will impact the
>> cache efficiency
>>
>> And the Ways are processing much more slowly than I would have
>> expected, namely:
>>
>> > Processing: Node(1329673k 43.6k/s) Way (3282k 0.12k/s) Relation(0
>> 0.00/s)
>>
>> Has anyone else imported this particular planet file and is the slow
>> Way loading to be expected after that error?
>>
>> Would I be better off in scrapping this slow load a!
nd going
back
>> another week to download the Planet and then apply the diffs to bring
>> it up to date?
>>
>> Lynn (D) - <a moz-do-not-send="true" href="http://aprsisce.wikidot.com">http://aprsisce.wikidot.com</a>/ - A user of OSM data
>>
>>
>>
>>
>><hr>
>> dev mailing list
>> <a class="moz-txt-link-abbreviated" href="mailto:dev@openstreetmap.org">dev@openstreetmap.org</a>
>> <a moz-do-not-send="true" href="http://lists.openstreetmap.org/listinfo/dev">http://lists.openstreetmap.org/listinfo/dev</a>
>
>
><hr>
> dev mailing list
> <a class="moz-txt-link-abbreviated" href="mailto:dev@openstreetmap.org">dev@openstreetmap.org</a>
> <a moz-do-not-send="true" href="http://lists.openstreetmap.org/listinfo/dev">http://lists.openstreetmap.org/listinfo/dev</a>
>
<hr>
dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:dev@openstreetmap.org">dev@openstreetmap.org</a>
<a moz-do-not-send="true" href="http://lists.openstreetmap.org/listinfo/dev">http://lists.openstreetmap.org/listinfo/dev</a>
</pre>
</blockquote>
</div>
<br>
-- <br>
Sent from my Galaxy Tab with Kaiten Mail.
</blockquote>
<br>
</body>
</html>