[OSM-dev] Full History Files

[Brett Henderson]

If anybody is interested I've put some sample full history changesets on 
the planet server to give people an idea of what is possible.
http://planet.openstreetmap.org/history

This may not be the best way of providing full history, but it is 
something I could produce on a daily/hourly/minute schedule like the 
current delta changesets.  The history files use an identical file 
format to the existing delta changesets, the only difference between the 
two osc formats being that with full history changesets it is possible 
for a single entity to appear multiple times (ie. one version per change 
in that time interval).  Perhaps I should create full history files with 
a different extension to avoid confusion, perhaps *.osh?

For downloading bulk history and avoiding downloading several hundred 
daily files, there are a couple of options:
* Merge multiple fill history osc files into a single file.  This can 
either be a straight concatenation, or can be re-sorted so that all 
versions of an entity are co-located.  I might have to tweak the 
existing merge tasks in osmosis to do this.
* Create a new tool that dumps into a full history osm format.  It would 
differ from existing planet files by having multiple versions of each 
entity (rather than just the latest) and each would have the visible 
flag set.

 From my perspective I prefer the osc format because it allows easier 
merging and manipulation of changes and is supported by osmosis.  But 
the great thing about this community is that we can make our own choices :-)

Having said all that, I won't start the full history daily process 
running just yet.  We need some more space on the planet server (which 
is already in planning), and it might be a good idea to let the existing 
0.6 transition settle down for a week or two to avoid overloading admins 
with problems.  It might take several days to process the full set of 
daily files and I'm not sure what kind of load it will place on the 
database server yet.

Anyway, happy to discuss.  And I'd like to hear if there are better 
alternatives or if people think this is a bad idea.

Brett