[OSM-dev] Limitation on downloading ways via /map API Call

James Mastros james at mastros.biz
Mon Jul 10 06:05:36 BST 2006

On Sun, Jul 09, 2006 at 09:12:43PM +0100, SteveC wrote:
> * @ 09/07/06 05:12:50 PM crschmidt at crschmidt.net wrote:
> > This seems to me to be ignoring the fact that we have a get_multi
> > function which does exactly what this SQL does, for a single way: given
> No, it's not ignoring it. It's making a balanced decision about calling
> the way function hundreds of times or trying to grab all that data at
> once. Up until today, as you know, it wasn't apparent that the
> group_concat functions would have a limit.

No, but it is apparent that group_concat doesn't really "fit" in the design
of SQL in general, and that joining all the disseperate records into a
string in the SQL server, and then splitting them back up in the Ruby code
is going to be sub-optimal.  The server is designed to give back multiple
records, not to do string manipulation.

> > This doesn't seem to me to be very hard to fix: all the SQL currently
> > uses the call_sql function, which uses get_connection, which currently is:
> Actually, you would want a pool of db connections not just one.

No, you want to have one db connection per simulintanious request.  There is
something of an issue of running out of mysql server handles.  However, it's
highly unlikely we'll ever reach that limit.  On the other hand, tearing up
the connection each time is always going to be bad, no matter what the
server load.  Note that different forks of the apache server are the only
way to have simulintanious requests, and there is one ruby interpreter per

Note, by the way, the first bullet point in
http://dev.mysql.com/doc/refman/5.1/en/tips.html -- the MySql manual,
7.2.19. Other Optimization Tips.  "Use persistant connections".

If in doubt, of course, benchmark, by all means.

   -=- James Mastros

More information about the dev mailing list