[OSM-dev] scaling

[Andy Allan]

On Sun, Jan 9, 2011 at 9:38 PM, SteveC <steve at asklater.com> wrote:

> So, what do you think? And if you agree it's worth doing, how do we achieve it either as individuals or the board or companies supporting it?

How do you eat an elephant? One bite at a time. (I'll come back to this later).

* To make what we have go faster, some parts of the api are worth
reimplementing in something faster than RoR. Matt's been doing that
with cgimap and that should be continued / expanded to cover more
performance-critical stuff.
https://github.com/zerebubuth/openstreetmap-cgimap
* We can scale our front ends and middleware servers already (tick!),
and it would be great make sure we stay far enough ahead of the curve
to cope with sudden influxes - i.e. have idle capacity
* The db server is the hard-to-resolve bottle-neck. I'm not entirely
convinced by the "we can't keep buying a bigger server" approach,
since lots of other people actually do just keep buying bigger servers
and it simplifies a lot of non-trivial things. When our
liquid-CO2-cooled db talks to its SAN over fibre channel and to the
middleware over multiplexed 10GigE connections, *then* we'll have to
think of something....
* ... OK, assuming we don't go that route, we can be much more
effective by scaling out read requests and worry about write scaling
separately
* Read-request scaling should be done by a XAPI equivalent that more
people can admin / hack on / understand. See e.g.
https://github.com/iandees/xapi-servlet . Other people have talked
about this approach on the thread.
* Write-replication is sort-of doable on postgres (see
http://en.wikipedia.org/wiki/Multi-master_replication#PostgreSQL ) but
will come back to the issues of bandwidth required for syncing - so we
need to consider how far we could get with e.g. 100 Mbps sync, 1Gb
sync, 10Gb sync etc between the masters

As a member of TWG though, my main concerns are more about redundancy,
availability, current draw (and fuse ratings, for those who know that
story) than scaling, but that's me and other people have different
priorities. I'll keep plugging away at them in the background since
it's not something that you can code or ask for suggestions about.

>From how to achieve things where you have X amount of time and Y
amount of dollars (and I suspect in Steve's case he has easier access
to dollars than time)
* My main concern is always about breadth of the technical community -
we're a small number of people compared to the size of the project.
Although it's not the way to get things done quickly, I like to focus
on getting more people involved rather than the existing people more
deeply involved (that tends to happen by itself!). I believe for every
100 developers who start committing some of them will be the next Tom
/ Grant / Matt / Frederik but it's impossible to tell who would be in
advance so it's my aim to get as many people involved as possible.
* I find running Hack weekends is a good use of a few hundred dollars
in getting people from keenly interested into actually committing code
* If you have time to spend, then focussing on the "what stops people
making their first commit" is the best place to consider. It's often
just rewriting some decaying documentation on the wiki about
installing some software
* Another good use of time is blogging, arguably more useful than
mailing list posts
* Doing the boring stuff (build systems, test cases, bug fixing,
clearing out old trac tickets) allows other people to get hooked on
doing the more interesting stuff
* Finally, paying people. I think the best way to deal with that isn't
to pay new people to do something in particular (e.g. code K for OSM
project M), it's to pay existing people to stop doing other non-OSM
things (like their day job). This lets them concentrate on what they'd
eventually get round to doing anyway, just faster, and you don't need
to worry about directing them if they are a die-hard contributor
already. (I'm slightly biased on this since, to a certain extent, this
is the position I'm in right now!)

So time goes into "getting started" documentation for coders, and
money goes into either bringing in new coders or taking care of things
to let core people concentrate.

Back to the elephant. So how do you eat it quickly? Get lots of
people, and everyone eats at the same time. Of all the ideas we should
do them all, now, as hard as we can.

Cheers,
Andy

> PS if it looks weird that I respond to certain emails and not others then that's because messages to, from or cc some of the trolls are automatically deleted and I don't see them. So even if you just cc them, I won't see your email. I highly recommend doing this.

What, cc'ing known trolls so that you don't see the replies? :-P