[Tile-serving] Switch2osm rewrite review needed

[Paul Norman]

> From: Frederik Ramm [mailto:frederik at remote.org]
> Sent: Saturday, May 10, 2014 5:43 AM
> Subject: Re: [Tile-serving] Switch2osm rewrite review needed
> 
> do we need "--no-install-recommends", will the stuff that is
> installed additionally really be such a burden?

The difference is 173 packages vs 141. The big difference is osm2pgsql
from the PPA pulls in openstreetmap-postgis-db-setup. You also get 
libgtk pulled in with postgis. For reference, the "recommended" 
packages are fakeroot libalgorithm-merge-perl libfile-fcntllock-perl 
munin-doc libnet-snmp-perl openstreetmap-postgis-db-setup postgis.

> * I am not comfortable with elevating osmctools to the mainstream (were
> they in switch2osm.org before?) but that might be a personal bias - I'm
> just so used to Osmosis and consider Osmosis the standard tool for
> updating. 

As for osmctools, they weren't in switch2osm before, but I decided to go 
with them for a few reasons, primarily the first two

- Their usage is simpler than osmosis, particularly for the one command 
  in this guide
- osmctools is packaged on Ubuntu 14.04, while the version of osmosis 
  packaged is 0.40.1
- osmupdate is faster than osmosis for this case apparently
- osmosis pulls in a huge dependency list with Java
- Building osmctools is far easier than building Osmosis

The only use for osmctools in this guide is an osmupdate on the planet
file before importing, which is a one-line command with osmupdate, and 
complicated with osmosis.

I'm not covering updating the DB at this point, and I honestly don't 
know what to recommend for that. I'm used to Osmosis myself, but the
--rrii and setup sequence number procedure is confusing, and having
to wrap the entire osmosis --rri --wxc | osm2pgsql command in a script
to handle crashes in osm2pgsql after state.txt is already updated removes
some of the advantages of osmosis.

> You also mention osmconvert for creating extracts

I've added a mention of osmosis to extracts

> * Is this overcommit thing a new requirement under Ubuntu 14.04? Because
> I've certainly never fiddled with it yet, and I run many a tile server.
> If it isn't essential, drop it.

On a machine where the number of processes times the cache exceeds available

memory it is important. This is going to be most machines without swap.

Without it, osm2pgsql will request memory for each process during the
pending 
ways stage and it will fail and fall back to 1 process.

> * For --hstore, you write "very minor speed penalty" but you should also
> mention a +10% size penalty on average.

Done.

> If you're doing an import for an
> area with lots of source/note/whatever tags then it can be significantly
> more - for example, a full --slim import for Japan takes 19 GB without,
> and 25 GB with -k (these numbers being the total amount of disk storage
> used by the PostGIS tablespace

I'd be interested if the differences for Japan are nearly as significant
with the new delete lines in default.style. They should significantly 
help with the import cruft in Japan.