[OSM-dev] OSM Date Formats

[Brett Henderson]

Jon Burgess wrote:
> I've updated the planet.c code to generate dates in the format:
>
>     2007-07-10T11:32:32Z
>
> Loading a small planet dump in old vs new date format using Osmosis
> produces the following timings:
>
> OLD:
>   real    12m44.875s
>   user    2m4.975s
>   sys     0m23.508s
>
> NEW:
>   real    10m47.347s 
>   user    0m43.055s
>   sys     0m2.443s
>
>
> A >50% drop in CPU utilisation by Osmosis, even more if you include the
> drop in system time as well. It is a shame that MySQL takes so long.
>
> 	Jon
>   
Excellent, thanks for that.

As for MySQL, yes it's a big problem.  Loading a full planet is 
incredibly painful now.  I'm open to suggestions on this one.  The 
MyISAM history tables are pretty fast, but the subsequent copying to the 
InnoDB tables takes forever.  I haven't timed it for 0.5 but 0.4 planet 
was taking around 8 hours at last check.  I've tried all sorts of things 
to speed it up including dropping and re-creating indexes, none of it 
seems to help much.  I'm wondering if I should create a new task to 
produce files that can be imported using the LOAD INFILE command but I 
don't know if that will help.