[OSM-dev] load-osm.sh

Paul Norman penorman at mac.com
Mon Oct 13 04:13:14 UTC 2014


On 10/12/2014 2:49 PM, Ilya Zverev wrote:
> Hi! Yesterday I had a simple task: there is a rendering server, which 
> is not to be minutely updated. So its PostgreSQL database doesn't need 
> temporary "slim" tables. But I cannot import an extract without 
> creating those, so basically I can import 3 times less data than 
> possible. 
It's worth noting that the --drop mode in osm2pgsql will delete the slim 
tables when done, and also not create the indexes on those tables. As 
those indexes are large, this is a substantial space savings in itself. 
--drop also will create more compact indexes for the rendering tables.

The space saved is more than a simple inspection of table and indexes 
sizes would cause you to expect. For an import, the key parameter is 
peak disk usage, which normally happens partway through the clustering 
and index creation phase. As the slim tables are dropped before this, 
they do not contribute. --disable-parallel-indexing will also reduce 
peak ram usage, though I am uncertain if the slim tables get dropped 
early enough with both this and --drop.

--slim --drop --disable-parallel-indexing should have a database not 
much larger than the final rendering tables.*

Of course, Zverev's technique has lesser space requirements, and 
probably faster, at the cost of requiring a second server.

* The final space required is (rendering tables + rendering indexes) 
while the largest space requirement is the maximum of (rendering tables 
+ slim tables), (rendering tables + largest rendering table again - 
largest rendering index)



More information about the dev mailing list