[OSM-dev] Faster loading with scabies

[Joachim Zobel]

Am Donnerstag, den 16.10.2008, 09:21 +1100 schrieb Brett Henderson:
> Do you have any numbers to show what sort of performance increase can
> be expected.  

I can load europe.osm into the current_* tables in less than 12 hours
(on my laptop - overpowered CPU, 5400 disk). 

> If it's significant I should add similar functionality into osmosis.

I did hope you would say that.

> The biggest problem I found wasn't the actual processing of INSERT
> statements, it was MySQL scaling non-linearly with the number of
> rows.  

I missed that one. I loaded europe osm with osmosis and found it too
slow. I then did all the speed tests with europe.osm.

Do you have any numbers concerning the scalability issue?

> MyISAM tables are very fast to import regardless of number of rows,
> but InnoDB seems to slow down as the number of rows increase.  I'm
> surprised loading with LOAD DATA INFILE fixes that.

LDI is probably the fastes way to get data into mysql. 

Sincerely,
Joachim