[OSM-dev] Google doesn't find new wiki urls

[Karl Guggisberg]

> It doesn't help that we have some ridiculously complicated pages like map features which are not only hugely complicated with masses of templates but 
> also get changed very frequently.

We are currently evaluating how Semantic MediaWiki can help us to manage map feature infos on the wiki (see [1] for ideas and [2] for dev playground).

We will still use templates, of course, even quite excessively, I'd say. In addition we will bring semantic inline queries into the game. This might have a significant impact on caching and I begin to wonder whether OSMs servers will be able to handle this kind of load. 

Does anybody have experience with installations of SMW with significant load?

-- Karl

[1] http://wiki.openstreetmap.org/wiki/Machine-readable_Map_Feature_list
[2] http://dev.openstreetmap.org/~edgemaster/semwiki/index.php/Main_Page


-----Ursprüngliche Nachricht-----
Von: dev-bounces at openstreetmap.org [mailto:dev-bounces at openstreetmap.org] Im Auftrag von Tom Hughes
Gesendet: Mittwoch, 14. Januar 2009 13:37
An: Ævar Arnfjörð Bjarmason
Cc: Dev Openstreetmap
Betreff: Re: [OSM-dev] Google doesn't find new wiki urls

 var Arnfj r  Bjarmason wrote:

> On Wed, Jan 14, 2009 at 10:36 AM, Tom Hughes <tom at compton.nu> wrote:
>
>> The googlebot is probably blocked as bots tend to kill the wiki due 
>> to the extreme crapness of mediawiki.
> 
> Bots happily hit Wikipedia all day without killing it, is the OSM wiki 
> set up equivalently? I.e. with a cache in front of it, memcache to 
> cache various things within MW etc?

Yes, we're setup a little differently.

We have one little Atom based machine. They have a metric buttload of database servers serving pages to ten metric buttloads of web servers fronted by about a thousand metric buttloads of squid caches.

Basically wikipedia have "solved" the problem of mediawiki being a crock by throwing ludicrous amounts of hardware at it.

It doesn't help that we have some ridiculously complicated pages like map features which are not only hugely complicated with masses of templates but also get changed very frequently.

Tom

--
Tom Hughes (tom at compton.nu)
http://www.compton.nu/

_______________________________________________
dev mailing list
dev at openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev