[OSM-talk] The British Museum Test

[David Earl]

On 24/11/2007 16:23, Robert (Jamie) Munro wrote:
> David Earl wrote:
>> BTW, the size of the planet file has grown so much since this was first 
>> done that it is now becoming impractical to process it. It took over 80 
>> hours last time, and seems to double every week. So I need to spend some 
>> time on it to bring this back to manageable time (I know what I need to 
>> do), but in the meantime, I can't update the index every week for now.
> 
> Have you considered using the Osmosis daily dumps to update your index
> incrementally? This would allow daily updates, and will probably take
> much less processing effort than using the whole planet each time.

Yes I was thinking about doing it incrementally, though I was thinking 
about using the weekly planet diffs. Doing it isn't a matter of just 
processing the diffs themselves though because a change to an object can 
make the index change for objects that haven't themselves changed - for 
example, say we have "Foo Road" on a couple of roads in a village, and 
someone realises one of them is wrong, so corrects it to "Bar Road". 
Wombat Road mustn't be deleted from the index, merely transferred to the 
other instance of Foo Road that remains, as well as adding Bar Road. I 
also have to spot nodes that change position leading to changes to the 
index for the highways that they are part of.

I know in outline the algorithm I need to cover all the cases, I just 
need to find time to hack at it.

David