[OSM-dev] Rantings about API 0.6
Iván Sánchez Ortega
ivan at sanchezortega.es
Tue Feb 10 00:32:35 GMT 2009
I have good news and bad news.
The good news is, I have a brand new bulk uploader that works with 0.6 and its
limitations (unless proven otherwise).
The bad news is, I've had a look at the SQL queries the Rails port is making
to the DB during a mass import, and all I could think is "WTF?!?"
So, let's have a look at this:
[2009-02-10 01:00:34.809017 #1664] User Load (0.000144) SELECT * FROM
`users` WHERE (`users`.`id` = 1)
[2009-02-10 01:00:34.810457 #1664] SQL (0.000144) SELECT `display_name`
FROM `users` WHERE (`users`.display_name = 'ivansanchez' AND `users`.id <> 1)
[2009-02-10 01:00:34.811241 #1664] SQL (0.000108) SELECT `email` FROM
`users` WHERE (`users`.email = 'ivan at sanchezortega.es' AND `users`.id <> 1)
WTF does this happen on every object (read: node) I upload? The API already
knows who I am the second I started to upload the diff!
[2009-02-10 01:00:34.863175 #1664] SQL (0.000244) SELECT `id` FROM
`current_nodes` WHERE (`current_nodes`.id IS NULL)
current_nodes.id is defined as NOT NULL. So, WTF?
[2009-02-10 01:00:34.864872 #1664] SQL (0.000258) SELECT `id` FROM
`changesets` WHERE (`changesets`.id = 44 AND `changesets`.id <> 44)
[2009-02-10 01:00:34.859181 #1664] Changeset Update (0.000314) UPDATE
`changesets` SET `num_changes` = 18443 WHERE `id` = 44
Couldn't this wait until the diff upload is complete?
(Rants kind of end)
So, while I'm no expert about Rails, I do suspect that all those extra
unnecessary queries are done automagically.
I do know OSM will have some brand new DB server (now with 0.1 more API!) that
should be able to handle the extra overhead, and that most of those queries
will hit data that will be cached anyway.
I also do think that there is a lot of room for optimizations (namely, removal
of DB overhead), so...
Is it worth to optimize the code for diff uploads?
Iván Sánchez Ortega <ivan at sanchezortega.es>
Q: What's the difference between Bell Labs and the Boy Scouts of America?
A: The Boy Scouts have adult supervision.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 197 bytes
Desc: This is a digitally signed message part.
More information about the dev