[OSM-talk-be] OSM in French and Dutch

Tue Aug 9 14:04:25 UTC 2016

You'll need a beafy box to be able to cope with a few clients,  not only
the DB size-on-disk is important, you'll want to cache those tiles as
well. So you will need double of that atleast.

The most important resources will be CPU and IO (and MEM).  Especially
the rendering part will take CPU/IO, the import and database need memory.

Importing the whole world means getting atleast 400Gigs in disks space,
on SSD's.  You should also test providers as some really suck on IO.  to
give you an idea, here are some basic benchmark all tested under some
load and on production systems (real world) :

Type(reads)                             Cached     Non-Cached

Laptop (2*SSD, 8GB)           10171.68 MB/sec   897.67 MB/sec
Linode (KVM) 4GB (now 8GB)     7881.80 MB/sec   552.31 MB/sec
Linode (KVM hypervisor) 32GB   8458.77 MB/sec   539.90 MB/sec
Rackspace VM (general purpose) 8638.16 MB/sec   298.06 MB/sec
Real server hardware (SATA)    9567.98 MB/sec   258.36 MB/sec
Private cloud(xen hypervisor) 10637.34 MB/sec   220.60 MB/sec
Hetzner VM 32GB                1703.88 MB/sec   153.84 MB/sec
Rackspace VM (8GB classic)     2556.96 MB/sec   100.62 MB/sec
Rackspace standard block store 8105.07 MB/sec    42.89 MB/sec

Hetzner for example, will not cut it with the 32GB server vs 8GB linode,
the tiles render incredibly slow. To compensate you should add a cache
layer. (I usually do this in nginx, it's the fastest for file serving).
 A linode with 8GB (after free upgrade from 4GB) will cope with
something like benelux sized data and will be about 5 times faster than
Hetzner.

If you use renderd, you can take advantage of getting more cores, then
you do more rendering in parallel.

It's a true maze.  Focus is IO + CPU, then get lots of memory, next in
line: try to get SSD's, then finally, make sure you have enough disks space.

The more memory, the more postgresql can run from memory. That speeds up
things as well.

I've set up a planet tile server once, and be prepared to import and
index for weeks.  It's just huge.  And the tested machines with the
specs are all too low for planet tile rendering.

If you get cheap stuff, there is always a catch.  So let performance
determine your price, don't let price determine performance.  Of course
it depends on how deep your pockets are.

Glenn

On 09-08-16 13:09, joost schouppe wrote:
> Hi Jonathan,
> 
> That would be extremely cool. Do you have an idea of the costs involved
> in setting this up for the area you mention and for the whole world?
> 
> I think if we can have a demo by SOTM, we might find someone to sponsor
> it there.
> 
> Joost
> 
> 2016-08-09 12:39 GMT+02:00 Jonathan Beliën <jbe at geo6.be
> <mailto:jbe at geo6.be>>:
> 
>     Hi everyone,____
> 
>     __ __
> 
>     Good point indeed.____
> 
>     __ __
> 
>     It’s not very complicated to generate a custom rendering. For
>     example, I did it for BeNeLux + France + Germany using imposm 3
>     (https://imposm.org/ ) and MapServer + MapProxy for rendering. ____
> 
>     __ __
> 
>     My rendering use « default » names but I could have used NL name or
>     FR names as needed.____
> 
>     __ __
> 
>     It works quite well and it’s not too complicated (even with a
>     recurring update). The only problem is the disk space those
>     « tiles » use on the server. It’s already quite big for the small
>     area I cover (BeNeLux + north of France + west of Germany) ; I can’t
>     imagine the number of tiles if we want to cover the whole world.____
> 
>     __ __
> 
>     I do not know how to organize that but we could have a « Belgium OSM
>     rendering FR » and a « Belgium OSM rendering NL » created, hosted
>     and maintained by « OSM.be ».____
> 
>     __ __
> 
>     Jonathan Beliën____
> 
>     GEO-6____
> 
>     __ __
> 
>     *De :*joost schouppe [mailto:joost.schouppe at gmail.com
>     <mailto:joost.schouppe at gmail.com>]
>     *Envoyé :* mardi 9 août 2016 11:38
>     *À :* OpenStreetMap Belgium
>     *Objet :* [OSM-talk-be] OSM in French and Dutch
>     *Importance :* Haute____
> 
>     __ __
> 
>     Hi,____
> 
>     __ __
> 
>     Someone asked on Twitter about a rendering of OSM in Dutch and
>     French to avoid the clutter of bilingual names in the standard
>     rendering.____
> 
>     __ __
> 
>     https://twitter.com/iciBrussels/status/762743820358418432
>     <https://twitter.com/iciBrussels/status/762743820358418432>____
> 
>     __ __
> 
>     The French render is easy, OSM France provides it. But how about a
>     Dutch rendering? Do you know of one?____
> 
>     __ __
> 
>     It might be cool to create a little webmap on OSM.be with the three
>     official languages. If you help me find a Dutch rendering, I can
>     make that (I've just learned the basics about leaflet).____
> 
>     __ __
> 
>     It looks rather easy to make a style with mapbox, but you need to
>     extract the data through Overpass for exotic languages like Dutch,
>     so it would be a bit of a job to keep that up to date.
>     ____
> 
>     __ __
> 
>     -- ____
> 
>     Joost @____
> 
>     Openstreetmap
>     <http://www.openstreetmap.org/user/joost%20schouppe/> | Twitter
>     <https://twitter.com/joostjakob> | LinkedIn
>     <https://www.linkedin.com/pub/joost-schouppe/48/939/603> | Meetup
>     <http://www.meetup.com/OpenStreetMap-Belgium/members/97979802/>____
> 
> 
> 
> 
> -- 
> Joost @
> Openstreetmap
> <http://www.openstreetmap.org/user/joost%20schouppe/> | Twitter
> <https://twitter.com/joostjakob> | LinkedIn
> <https://www.linkedin.com/pub/joost-schouppe/48/939/603> | Meetup
> <http://www.meetup.com/OpenStreetMap-Belgium/members/97979802/>
> 
> 
> _______________________________________________
> Talk-be mailing list
> Talk-be at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-be
> 