[OSM-talk] Installing your own tileserver on Ubuntu

[Joseph Reeves]

Hi Kai,

>The default Ram cache size is 800Mb. You can increase it with the -C
>parameter of osm2pgsql. But given that your netbook probably doesn't
>have that much ram, I am not sure increasing that option is such a good
>idea.

-C only has an effect in slim mode, unless I'm wrong? As I was hoping
to append data to my db, I'm running osm2pgsql in full fat mode.
Thanks for the --cache-strategy tip - I got the import working with
the sparse option. It seems to be working surprisingly quickly in
fact.

Of course, as you said, getting the data into the db is one thing, but
actually using it is another matter.  The netbook is now rendering
tiles for a large strip across Europe - this often takes a while to
get tiles created, but I think it should work for what I need it for.
If the load gets too high I can always empty the db and add extracts
as I need them. Before that, however, I think I'll try and find any
database optimisations that might exist.

Thanks again for all this,

Cheers, Joseph



On 18 October 2011 18:34, Kai Krueger <kakrueger at gmail.com> wrote:
> On 10/18/2011 10:48 AM, Joseph Reeves wrote:
>>> It is possible that one could catch the errors in slim mode and then only do the expensive diff processing for those node / ways that are >duplicate in the extracts.
>>
>> Interesting, although I think this is beyond the limits of my OSM
>> skills.
>
> Yes, that comment was more directed at my self that I should look into
> that, or if any other dev of osm2pgsql gets to it first... ;-)
>
>  Unfortunately Austria seems to be beyond the capabilities of
>> my netbook; an import without --slim gives the error:
>>
>> Node cache size is too small to fit all nodes. Please increase cache size
>>
>> Presumably a slim import would help, but this would then fail because
>> of overlapping ways... I can't Google up anyone suffering that error
>> message before; I guess nobody else is trying to get a number of
>> European countries into a db on their netbook...
>
> You will likely not find that error message on google yet, as if I am
> not mistaken, I only commited that error message two days ago.
>
> The default Ram cache size is 800Mb. You can increase it with the -C
> parameter of osm2pgsql. But given that your netbook probably doesn't
> have that much ram, I am not sure increasing that option is such a good
> idea.
>
> Given that you can't fit Austria into 800Mb, I suspect you are using the
> 32bit version of osm2pgsql. It defaults to the old (for extracts
> inefficient) cache allocation strategy. You can try using the
> "--cache-strategy" option and set it to either optimized or sparse. That
> should be more efficient for extracts than the default method. In the
> optimized option, you might run out of virtual memory on 32 bit though.
>
> But you might have troubles with Austria on a netbook anyway, as it
> might still be too resource intense.
>
> Kai
>
>>
>> Thanks again,
>>
>> Joseph
>>
>>
>>
>>
>> On 18 October 2011 16:59, Kai Krueger <kakrueger at gmail.com> wrote:
>>> On 10/18/11 9:31 AM, Joseph Reeves wrote:
>>>>
>>>> Hi Kai,
>>>>
>>>>> The pre-rendered tiles are stored in /var/lib/mod_tile/default. You can
>>>>> simply delete those files and they will automatically get rerendered the
>>>>> next time you view them.
>>>>
>>>> Great, thanks, that's working great.
>>>>
>>>>> I have seen that you appear to need to restart renderd (sudo
>>>>> /etc/init.d/renderd restart) after a new import, as it otherwise appears
>>>>> to
>>>>> still use old data (It is kind of odd, so I might have the wrong
>>>>> impression
>>>>> here).
>>>>
>>>> Sorry, I should've said in my previous email - I was assuming this to
>>>> be the case. Out of interest, is there any log output from renderd?
>>>
>>> sudo tail -f /var/log/syslog
>>>
>>> should give you some output of what renderd is doing, including which tiles
>>> it is rendering and when it has completed them
>>>>
>>>> I'm running this on a netbook so tiles take a while to be produced;
>>>
>>> I'd imagine a netbook might struggle a little with rendering tiles on the
>>> fly... ;-) But eventually they should be done.
>>>>
>>>>  it
>>>> would be interesting to see some logs so that I know it's all working
>>>> as I expect.
>>>>
>>>>> However, what you are trying to do is as far as I know not supported by
>>>>> osm2pgsql. Although it seems to be a much requested feature, I don't
>>>>> think
>>>>> osm2pgsql currently handles importing of multiple extracts. The --append
>>>>> option doesn't really do what you would think it does.
>>>>
>>>> I tried the append flag and got an error about an already existing way
>>>> - it would be good if osm2pgsql would simply ignore any ways that
>>>> already exist in the database. I re-ran osm2pgsql without the --slim
>>>> option, however, and the import was successful. I currently have
>>>> Bulgaria and Romania working on my netbook :)
>>>
>>> Interesting that the non-slim mode works with appending multiple extracts.
>>>
>>> It is possible that one could catch the errors in slim mode and then only do
>>> the expensive diff processing for those node / ways that are duplicate in
>>> the extracts.
>>>>
>>>> Am trying to re-import Turkey now, then onwards with bits of Europe!
>>>> If it all works out do you mind if I do a bit of wiki fiddling on your
>>>> instructions?
>>>
>>> No go ahead and improve the instructions
>>>>
>>>> Thanks again,
>>>>
>>>> Joseph
>>>>
>>>
>>>
>
>