[Tile-serving] Tiles aren't rendering

[Sarah Hoffmann]

On Tue, Apr 23, 2019 at 09:49:38PM -0300, JunkMail wrote:
> Hi,
> 
> That's a good point. Is there any way to regenerate the indexes that
> osm2pgsql normally generates?

Nothing out of the box.

My suggestion would be to quickly import a tiny extract into
a different database using exactly the same paramters (minus
flat node file, that makes no difference and only needs space).
Then check the indexes that exist there and add to the planet
database any indexes that are missing.

You should also check if there are any tables left that have
a _tmp suffix. If that is the case, you are really in trouble.
osm2pgsql copies the data into temporary tables first and then
moves them to the final table while geographically sorting
on the way. That is also one of the reasons why so much space
is needed during import. The tables exist twice temporarily.
osm2pgsql does try to remove any unnecessary data as early as
possible but that kind of duplication cannot be avoided for
the sorting. It can be minimized by disabling multithreading
during indexing (--disable-parallel-indexing). Then one table
after the next will be copied instead of all in paralllel.

If you are tight on disk space next time, rather have a look
at zfs. It supports transparent compression and people reported
that it plays well with osm2pgsql imports.

Sarah

> 
> On 2019-04-23 5:23 p.m., Sarah Hoffmann wrote:
> > Hi,
> > 
> > given that you have been killing random Postgres processes
> > while osm2pgsql was in the indexing phase, it is fair to assume
> > that some of the processes were actually doing important stuff
> > and you are now missing a couple of indexes.
> > 
> > I would have expected that postgres returns an error in such
> > a case and osm2pgsql errors out but we might not catching
> > exceptions in the worker threads properly. Something to check.
> > 
> > There surely is a warning in the code somewhere along the
> > lines of: "If one of [the worker threads] has an error, all
> > other will finish first, which may take a long time." So if
> > the process killing did something bad you wouldn't have seen
> > the effect immediately.
> > 
> > Sarah
> > 
> > On Tue, Apr 23, 2019 at 03:47:39PM -0300, Stephen D wrote:
> > > Hi,
> > > 
> > > My tile server is not rendering tiles. I had it working with Canada(imported
> > > w/o a flat file), but after dropping the database and re-importing the whole
> > > planet(with a flat file) it no longer works.
> > > 
> > > The command I used to import was: osm2pgsql -d gis --create --slim  -G
> > > --hstore --tag-transform-script
> > > ~/src/openstreetmap-carto/openstreetmap-carto.lua --flat-nodes
> > > ~/data/flatnodes.bin  -C 6000 --drop --number-processes 8 -S
> > > ~/src/openstreetmap-carto/openstreetmap-carto.style
> > > /home/stephen/planet-latest.osm.pbf
> > > 
> > > When requesting a tile, this shows up in the log file:
> > > 
> > > Apr 23 18:37:15 osm-tilemap renderd[1488]: DEBUG: Connection 0, fd 7 closed,
> > > now 0 left
> > > Apr 23 18:39:21 osm-tilemap renderd[1488]: DEBUG: Got incoming connection,
> > > fd 7, number 1
> > > Apr 23 18:39:21 osm-tilemap renderd[1488]: DEBUG: Got incoming request with
> > > protocol version 2
> > > Apr 23 18:39:21 osm-tilemap renderd[1488]: DEBUG: Got command RenderPrio
> > > fd(7) xml(ajt), z(1), x(1), y(1), mime(image/png), options()
> > > Apr 23 18:39:51 osm-tilemap renderd[1488]: DEBUG: Connection 0, fd 7 closed,
> > > now 0 left
> > > 
> > > After 30 seconds it times out. No tiles appear in the cache directory, even
> > > after several minutes. The CPU reaches 20% per core for all cores, and disk
> > > usage is around 200MB/s read. Both the CPU and disk are used by the
> > > postgresql process.
> > > 
> > > I am using apache2, mod_tile, mapnik, and renderd.
> > > 
> > > Thanks,
> > > 
> > > Stephen
> > > 
> > > 
> > > _______________________________________________
> > > Tile-serving mailing list
> > > Tile-serving at openstreetmap.org
> > > https://lists.openstreetmap.org/listinfo/tile-serving