[OSM-dev] Faster loading with scabies
Sascha Silbe
sascha-ml-gis-osm-dev at silbe.org
Thu Oct 16 20:07:17 BST 2008
On Thu, Oct 16, 2008 at 09:21:54AM +1100, Brett Henderson wrote:
> The biggest problem I found wasn't the actual processing of INSERT
> statements, it was MySQL scaling non-linearly with the number of rows.
> MyISAM tables are very fast to import regardless of number of rows,
> but
> InnoDB seems to slow down as the number of rows increase. I'm
> surprised
> loading with LOAD DATA INFILE fixes that.
Just a quick stab in the dark as I do know almost nothing about osmosis
and MySQL:
If you create the indices before loading data, they will be constantly
rebuilt as you add each row => would explain the exponential slow-down.
The PostgreSQL manual has a section about optimizing database imports
giving some useful tips & tricks; I'd guess the same is true for the
MySQL one.
If osmosis already defers index creation (including referential
constraints which usually build indices implicitly), just ignore this
mail. :)
CU Sascha
--
http://sascha.silbe.org/
http://www.infra-silbe.de/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 481 bytes
Desc: Digital signature
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20081016/6fc72d66/attachment.pgp>
More information about the dev
mailing list