[OSM-dev] Full History Files
brett at bretth.com
Thu Apr 23 02:04:48 BST 2009
If anybody is interested I've put some sample full history changesets on
the planet server to give people an idea of what is possible.
This may not be the best way of providing full history, but it is
something I could produce on a daily/hourly/minute schedule like the
current delta changesets. The history files use an identical file
format to the existing delta changesets, the only difference between the
two osc formats being that with full history changesets it is possible
for a single entity to appear multiple times (ie. one version per change
in that time interval). Perhaps I should create full history files with
a different extension to avoid confusion, perhaps *.osh?
For downloading bulk history and avoiding downloading several hundred
daily files, there are a couple of options:
* Merge multiple fill history osc files into a single file. This can
either be a straight concatenation, or can be re-sorted so that all
versions of an entity are co-located. I might have to tweak the
existing merge tasks in osmosis to do this.
* Create a new tool that dumps into a full history osm format. It would
differ from existing planet files by having multiple versions of each
entity (rather than just the latest) and each would have the visible
From my perspective I prefer the osc format because it allows easier
merging and manipulation of changes and is supported by osmosis. But
the great thing about this community is that we can make our own choices :-)
Having said all that, I won't start the full history daily process
running just yet. We need some more space on the planet server (which
is already in planning), and it might be a good idea to let the existing
0.6 transition settle down for a week or two to avoid overloading admins
with problems. It might take several days to process the full set of
daily files and I'm not sure what kind of load it will place on the
database server yet.
Anyway, happy to discuss. And I'd like to hear if there are better
alternatives or if people think this is a bad idea.
More information about the dev