[Tile-serving] [osm2pgsql] Unnecessary flat-nodes with enough RAM (#22)

Kai Krueger kakrueger at gmail.com
Tue Apr 30 18:05:34 UTC 2013


On 04/30/2013 11:58 AM, Frederik Ramm wrote:
> Hi,
>
> On 30.04.2013 19:51, pnorman wrote:
>> It's been said that you could almost use /dev/null for a flat-nodes file
>> if you have a large enough -C value and don't plan to update the
>> database. An option that allows you to use no file at all for flat nodes
>> would be useful for when someone doesn't plan to update diffs and has
>> enough memory. It would save on sequential writes and save on disk
>> space. The latter is useful when using SSDs.
>
> Why would you use --slim under these circumstances at all?
Because the node cache is only a small portion of what needs to fit into 
memory in non-slim mode. The node cache needs about 16GB currently. The 
way cache needs a lot more if smaller extracts are something to go by.

The reason it currently doesn't work with /dev/null is that  at the 
forking stage of "going over pending ways" the file is reopened and the 
header is read in to determine the highest node id stored in the file. 
But with one or two lines commented out of the code, it does work.

However, simply writing the file sequentially isn't really an issue. 
Even on a single conventional disk, you can write a 16GB file typically 
in < 5 minutes. The node stage (for a full planet import) is entirely 
CPU bound, even on a very powerful single core CPU.

Kai
>
> Bye
> Frederik
>




More information about the Tile-serving mailing list