OSM's future. Was: Re: [Openstreetmap] OSM Performance is terrible

[Christopher Schmidt]

On Thu, Feb 02, 2006 at 12:28:54PM +0000, SteveC wrote:
> * @ 02/02/06 12:24:24 PM crschmidt at crschmidt.net wrote:
> > 50GB a month is very little in the US: That type of bandwidth can be
> 
> /me nods
> 
> but multiply that per machine and the stats graphs.

Even still, I'm currently dealing with 500GB/month in traffic - much of
it bittorrent, so I'm nowhere near informative with OSM's model, I get
that - and my hosts don't bug me.


> > bought for cheap. I'm currently paying $100/month for a dedicated server
> > with 1000GB/month bandwidth, just for comparison, and that's with a dedicated
> > server inluded, so presumably at least as good for colo.
> 
> I'd thought about that, but figured donated equipment at a university
> was going to be easier.

I think that it may be time to move OSM into a next stage. Especially
given the OS's attitude towards mapping projects, as you described in a
previous email, I think that looking to the people who may depend on
them - universities - to host things for free is risky. I can't imagine,
for example, wikipedia being hosted by a University. I know Google was
to start with, but they had to move out eventually, and with the number
of contributors OSM has, maybe this is a sign that it's time.

In other words, in my opinion, it may be time to take this effort, look
for some funding from various sources to cover recurring monthly costs,
and make the leap into a 'real' space with service level agreements and
so on. Taking the server down to do maint is something that is becoming
increasingly more troublesome as OSM gets a constant string of edits --
any time the server is down is time these things can't be updated. 6
months from now, moving the machines again when they outgrow another
university's goodwill, will be even more trouble than it might be now.

OSM has significant needs beyond a level that can be accomodated without
thought. These needs may be best met by someplace that is willing to
sign a contract to maintain a level of service, something that very few
people offering 'free' hosting will be willing to do.

Then again, I'm mostly an observer, so I may be wrong. But I'd love to
support OSM in this way if I can. 

> > In my experience, bandwidth in Europe and the UK seems to be more
> > expensive. I know that OSM is a mostly UK project, but maybe a US based
> > host could provide more bang for the buck, even with slightly higher
> > ping times or whatever. I'm not sure that bandwidth is the issue here,
> > of course, but if "50GB" is a lot in the current setup, I think the
> > current setup might not be optimal.
> 
> That 50Gb is all (well, a lot) on the fly generated images, not static
> content. Hence the load doesn't look like a static machine.

Load is definitely something I understand -- I don't question that the
hardware you mention is needed, but assuming all the machines are in the
same rackspace, the amount of processing is irrelevant to the actual
bandwidth.

In other words, bandwidth should be a target number that can be met
irrelevant of the hardware that's running behind the switch, unless I'm
misunderstanding.

> > Is there any way to get OSM set up as a non-profit organization for tax
> > purposes? I'd be happy to donate some money in the near future towards a
> > new equipment drive, but would like it even more if I could write it
> > down as a tax deduction.
> 
> I'm working on it :-)

Cool.

I'd love to see OSM do a pledge drive. Decide what hardware is needed to
make a workable system of machines, then create a wikipedia-like pledge
drive. Given the attitude of "we'd love to help, but we can't", perhaps
a monetary donation would be a more fitting request from users of the OS
data. (Anonymous to the greatest extent allowable by law...)

-- 
Christopher Schmidt
Web Developer