[Tile-serving] [openstreetmap/mod_tile] Replace select() by poll(). (#239)

[Tom Hughes]

This is a rebased version of #208 given the original requestor hasn't responded to the request to rebase and we are now running into this on tile.openstreetmap.org:

select() has a well-known limit and the maximum file descriptor
that can be used (FD_SETSIZE). On Linux it is in most cases 1024.

Our renderd uses a lot of planet tiff files, so it had already
more than 1600 FDs for tiff files open. That means any new FD gets a
number bigger than that and using such FDs in select() leads to crashes
(because select() uses a fixed size bitmap).

The same can happen in mod_tile if the surounding web server is very
busy, altough it is less likely to happen there.

Changes in src/mod_tile.c:
- include poll.h
- poll timeout is an int containing miliseconds
- use "s > 0" instead of "s == 1" as success
  (although it should never be >1)
- different log message for timeout and error case

Changes in src/daemon.c:
- remove include for sys/select.h
- replace connections array by new array pfd used in poll()
- include special FDs (exit and listen) at index 0 and 1 in this array
- no longer reorganize array while iteratig through it.
  That would have led to items being skipped.
  Instead mark array slots with fixed connections with
  a negative FD. These are automatically skipped by poll.
  Reuse these slots later when new connections come in.
- rename num_connections to num_cslots (initial segment
  of the array that poll() should check) and
  num_conns (number of slots in this segment, that are
  actually used by connections; this number is only
  used in logging).
- slightly enhance debug log lines
You can view, comment on, or merge this pull request online at:

  https://github.com/openstreetmap/mod_tile/pull/239

-- Commit Summary --

  * Replace select() by poll().

-- File Changes --

    M src/daemon.c (94)
    M src/mod_tile.c (23)

-- Patch Links --

https://github.com/openstreetmap/mod_tile/pull/239.patch
https://github.com/openstreetmap/mod_tile/pull/239.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/mod_tile/pull/239
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20210607/96996290/attachment-0001.htm>