[OSM-dev] Osmosis Replication Statistics

[Martijn van Oosterhout]

On 8/22/07, Brett Henderson <brett at bretth.com> wrote:
> Now, back to the huge planet file.  I have no idea why the planet file
> is so big.  My first thought was that my 12-month change application
> process was wrong so I took a snapshot at 20070101 to verify the
> results.  The files were *almost* identical in size.  The differences
> are contained in the file attached to this email.

There have been manual manipulations of the DB in the past: for
example the TIGER import was removed by hand, maybe the history
entries were neglected?

> So two problems to focus on:
> 1. Why is the planet file is so big?
> 2. Why do I have differences between my snapshotted 20070101 planet and
> my derived 20070101 using 12 months of changes?
>
> Problem 1.
> I have no idea what is causing this huge planet.  My only thought is
> that perhaps data exists in the history tables that doesn't exist in the
> current tables.  I'm really not sure what's going on here.  I need to
> look into this further.

Can you not diff the most recent planet against your generated
version, that should at least tell you where to look. Remember, even
simple things like using <tag></tag> instead of <tag/> can blow up the
file incredibly.

> Problem 2
> I've examined a random sample of the changes between my two 20070101.osm
> files.  For each change I examined the history of the entity in
> question.  In every case I've checked the change can be explained by the
> fact that the two most recent history rows (as of beginning 2007) have
> identical timestamps.  This means my queries sometimes return one row,
> sometimes the other depending on the particular query characteristics.
> I don't think there's much I can do about this.  Given that it is a very
> small set of changes, it is probably something we can live with and fix
> on a case by case basis as problems are picked up.

Got some examples?

Have a nice day,
-- 
Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/