[OSM-dev] history & disk space

Robert (Jamie) Munro rjmunro at arjam.net
Thu Sep 27 16:22:56 BST 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Steve Coast wrote:
> When I last looked when you added a object the server
> 
> a) inserts the data in to the current nodes table
> b) inserts the same data in to a node history table
> 
> When you update it
> 
> a) updates the data in the current nodes table
> b) inserts the same data in to a node history table
> 
> So we use at least twice the storage space. We could change this so  
> that history is only added on an update and not by default on  
> creation. It creates overhead in that when you look up the history of  
> an object it needs to check both the history and current table (which  
> I'm guessing is a rare API call), but saves some large number of Gb  
> in space.

Once osmosis is running, it won't just be a rare api call, it will be an
hourly (or even more frequently) used db query.

If you add a "current" column to the history tables, and put it at the
start of some of the indexes, it may be possible to run all queries
directly from the history tables, and the current tables become
redundant. In postgres and other database systems, you can create a
conditional index which will only index the current rows for this sort
of purpose. You can use rules to ensure that only one record is current
at a time. Also, you can use views to emulate hide this stuff from the
queries and even make the view updatable with triggers, so the front end
just makes updates as though there was no history at all, and the DB
handles it all.

Robert (Jamie) Munro
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG+8rMz+aYVHdncI0RAgfXAKCGslN6AIiqxvenXNvw+BQpoEYaoACcCoIl
ss4fso4N0yioqsikuzfPn5g=
=J0Kj
-----END PGP SIGNATURE-----




More information about the dev mailing list