[Tile-serving] Tiles aren't rendering

[Stephen - junkmail]

I decided to wipe the database and start fresh. I'm pretty sure my SSD 
is dying anyway. For now I will stick to a single country on a hard 
drive. Maybe I will buy a 1TB SSD and attempt a planet-wide tile server 
again in the future.

Thank you for your help!

On 2019-04-24 4:31 a.m., Sarah Hoffmann wrote:
> On Tue, Apr 23, 2019 at 09:49:38PM -0300, JunkMail wrote:
>> Hi,
>>
>> That's a good point. Is there any way to regenerate the indexes that
>> osm2pgsql normally generates?
> Nothing out of the box.
>
> My suggestion would be to quickly import a tiny extract into
> a different database using exactly the same paramters (minus
> flat node file, that makes no difference and only needs space).
> Then check the indexes that exist there and add to the planet
> database any indexes that are missing.
>
> You should also check if there are any tables left that have
> a _tmp suffix. If that is the case, you are really in trouble.
> osm2pgsql copies the data into temporary tables first and then
> moves them to the final table while geographically sorting
> on the way. That is also one of the reasons why so much space
> is needed during import. The tables exist twice temporarily.
> osm2pgsql does try to remove any unnecessary data as early as
> possible but that kind of duplication cannot be avoided for
> the sorting. It can be minimized by disabling multithreading
> during indexing (--disable-parallel-indexing). Then one table
> after the next will be copied instead of all in paralllel.
>
> If you are tight on disk space next time, rather have a look
> at zfs. It supports transparent compression and people reported
> that it plays well with osm2pgsql imports.
>
> Sarah
>
>> On 2019-04-23 5:23 p.m., Sarah Hoffmann wrote:
>>> Hi,
>>>
>>> given that you have been killing random Postgres processes
>>> while osm2pgsql was in the indexing phase, it is fair to assume
>>> that some of the processes were actually doing important stuff
>>> and you are now missing a couple of indexes.
>>>
>>> I would have expected that postgres returns an error in such
>>> a case and osm2pgsql errors out but we might not catching
>>> exceptions in the worker threads properly. Something to check.
>>>
>>> There surely is a warning in the code somewhere along the
>>> lines of: "If one of [the worker threads] has an error, all
>>> other will finish first, which may take a long time." So if
>>> the process killing did something bad you wouldn't have seen
>>> the effect immediately.
>>>
>>> Sarah
>>>
>>> On Tue, Apr 23, 2019 at 03:47:39PM -0300, Stephen D wrote:
>>>> Hi,
>>>>
>>>> My tile server is not rendering tiles. I had it working with Canada(imported
>>>> w/o a flat file), but after dropping the database and re-importing the whole
>>>> planet(with a flat file) it no longer works.
>>>>
>>>> The command I used to import was: osm2pgsql -d gis --create --slim  -G
>>>> --hstore --tag-transform-script
>>>> ~/src/openstreetmap-carto/openstreetmap-carto.lua --flat-nodes
>>>> ~/data/flatnodes.bin  -C 6000 --drop --number-processes 8 -S
>>>> ~/src/openstreetmap-carto/openstreetmap-carto.style
>>>> /home/stephen/planet-latest.osm.pbf
>>>>
>>>> When requesting a tile, this shows up in the log file:
>>>>
>>>> Apr 23 18:37:15 osm-tilemap renderd[1488]: DEBUG: Connection 0, fd 7 closed,
>>>> now 0 left
>>>> Apr 23 18:39:21 osm-tilemap renderd[1488]: DEBUG: Got incoming connection,
>>>> fd 7, number 1
>>>> Apr 23 18:39:21 osm-tilemap renderd[1488]: DEBUG: Got incoming request with
>>>> protocol version 2
>>>> Apr 23 18:39:21 osm-tilemap renderd[1488]: DEBUG: Got command RenderPrio
>>>> fd(7) xml(ajt), z(1), x(1), y(1), mime(image/png), options()
>>>> Apr 23 18:39:51 osm-tilemap renderd[1488]: DEBUG: Connection 0, fd 7 closed,
>>>> now 0 left
>>>>
>>>> After 30 seconds it times out. No tiles appear in the cache directory, even
>>>> after several minutes. The CPU reaches 20% per core for all cores, and disk
>>>> usage is around 200MB/s read. Both the CPU and disk are used by the
>>>> postgresql process.
>>>>
>>>> I am using apache2, mod_tile, mapnik, and renderd.
>>>>
>>>> Thanks,
>>>>
>>>> Stephen
>>>>
>>>>
>>>> _______________________________________________
>>>> Tile-serving mailing list
>>>> Tile-serving at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/tile-serving