[OSM-dev] Osmosis import to postgis - speed

Brett Henderson brett at bretth.com
Sun Dec 7 00:15:57 GMT 2008


Ah, sorry.  I missed this email when I wrote my previous replies :-)

It sounds like you're well on the way.  Hopefully my other emails help 
somewhat.  For example, you don't need to alter that variable in code, 
you can specify it on the command line with the inMemoryLinestring=yes 
option.  And there are already scripts to add the missing columns.

If import speed is an issue, try out the dump version of the pgsql 
writer task, it should speed things up further.  I'd be surprised if you 
couldn't knock off another few hours at least.

S Knox wrote:
> For information, setting the variable DEFAULT_IN_MEMORY_LINESTRING in 
> PostgreSqlDatasetWriterFactory.java to true caused the import to run 
> much faster - about 1.5 hours. It may also have helped that I set the 
> geometry correctly with the new geometry column, i.e:
> SELECT AddGeometryColumn('ways', 'linestring', 4326, 'LINESTRING', 2);
>
> Which I hadn't done before.
>
> Regards
>
> Steve
>   
> -----------------------------------------------------------------------------------------------------------------------
> I'm using Osmosis to import a recent UK extract of the planet file 
> into a database, and I used the simple schema here: 
> http://svn.openstreetmap.org/applications/utils/osmosis/script/pgsql_simple_schema_0.6.sql
>
> This took around 1 hour.
>
> I then found that this schema didn't give way geometries so re-ran the 
> query, adding a geometry column 'linestring' so that way objects would 
> be created.
>
> This import has taken 24 hours so far and isn't yet finished. Is this 
> normal or has something gone wrong? It would be useful to have some 
> sort of counter to find out how far into the process osmosis is - for 
> example roughly what line of the XML it is on, as a percentage of the 
> whole file.
>
> Admittedly I am going against advice and using .bz2 files, but it 
> appears postgresql is causing all the overhead.
>
> Thanks
>
> Steve
>
>
>  
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>   





More information about the dev mailing list