[OSM-dev] planet.openstreetmap.org breakage

[Brett Henderson]

Stefan de Konink wrote:
> On Thu, 19 Mar 2009, Brett Henderson wrote:
>
>   
>> Osmosis will provide the capability to do this from 0.6 onwards.  If
>> disk space isn't an issue on the planet server I'll set it up as soon as
>> I get time after 0.6 goes live.  It has the ability to go back in time
>> and produce full history diffs for any time interval in the same way it
>> currently has the ability to produce delta diffs for any time interval.
>> Again, if disk space isn't an issue I can create them for the life of OSM.
>>     
>
> You will query each changed object? Don't you think it would be nifty to
> have these history tables before we move to 0.6? Instead of having now
> half a year worth of planet files?
>   
I'm not sure I understand your questions.

Osmosis already has the ability to produce changeset files for any point 
in time, the only issue is that it merges multiple changes for a single 
object into a single change.  For example if a node gets modified twice 
in a given day, then only a single modification record will be written 
to the daily change file.  The 0.6 mysql change reading task adds a new 
option that allows the full history for that period to be written to the 
change file.  The code for querying the database is already proven and 
there is basically one query per table so it scales well.  The history 
tables are indexed by timestamp and the query response times are 
proportional to the amount of data being read.  So no, I won't query 
each changed object individually but yes I can extract every single 
change that has ever been made and produce what I *think* you're looking 
for.

I don't understand your question about history tables.  I'm not creating 
tables anywhere.  If I dump complete history it will be into xml files 
that people can download and process as they please.  Among other things 
it would let people visualise OSM changes over time with all data in a 
0.6 format.

As for having half a year worth of planet files, I'm not in any hurry to 
fix that particular problem.  0.6 should be out soon, and therefore I've 
only made changes to the 0.6 tasks and not the 0.5 tasks.  Planet files 
have been around for several years now, another few months won't mean 
the end of the world.  If somebody wishes to send me a patch to osmosis 
to extract full history from a 0.5 mysql database then I'll commit it.

Brett