[OSM-dev] Too many slow queries in db

Tom Hughes tom at compton.nu
Tue Sep 4 09:19:36 BST 2007


In message <20070904094707.C58933-100000 at xs2.xs4all.nl>
        Stefan de Konink <skinkie at xs4all.nl> wrote:

> On Tue, 4 Sep 2007, Tom Hughes wrote:
>
>> Do you know how the proxy handle allocation of new values to
>> auto increment columns? That seems like a major problem with
>> splitting up the data to me?
>
> Now you are describing INSERT queries. I hope you are not saying that this
> type of query is the reason of the current slowness. I'm not aware of the
> internals of *any* MySQL product, but implementing a loadbalancing system
> for at least fail over is pretty common these days.

Oddly enough we're doing quite a lot of INSERT queries, so they are
fairly important to any solution.

>> To be honest, somebody pointed out this proxy thing before and I did
>> have a look at it then and found the web page very hand-wavy and
>> lacking in any serious detail or documentation. It looks like it
>> is a bit new and maybe not very mature? or is that just me?
>
> I have some trust in code claimed to be 'Enterprise', but you should
> benchmark it, anything that you install anyway :)

Yeah. I'll just use those few spare days I left lying around to do
some benchmarking.

Look - it is quite simple. I am spending all my spare time firefighting
the current setup. I have no time to design and benchmark vast new
infrastructures. If you're so interested in it and think you make it
work then please be my guest - go build a prototype and benchmark it
and come back and tell us how much better it is.

>> If the only reason you think it is useful is as a way to split the
>> data over multiple physical machines, then I have to say that I'm
>> not sure we've reached that point yet - we may do with all the Tiger
>> data but it sounds like Jon is looking at that and in the meantime
>> there are other things I think we can do.
>
> Keeping everything on one machine 'because you can' is not the way to look
> to the future. I'm a very pro for geo splitting data (distribution wise)
> but still even still solution doesn't make the system *scale*. It is only
> divide and conquor and not plug and play.

See above. I look forward to seeing your design and the code to 
implement it.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/




More information about the dev mailing list