[OSM-dev] Osmarender not always showing latest data

milenko at king-nerd.com milenko at king-nerd.com
Tue Dec 16 00:57:43 GMT 2008


I haven't figured out how the server got out of sync with the main api yet.  My best guess is that I somehow missed part of a day or a few hours when I initially brought the server online.  If I remember correctly the timeframes are about the same.

The only issue I've had with --rci was during the api outtage last weekend.  The main server produced a 0-byte minute diff during the outtage and osmosis wouldn't scan past that file.  I had to apply the hourly diff and then go from there.

-Jeremy

Original Message -----------------------
It sounds like a great idea to ensure all servers have all data, however 
I'm curious how any could get out of sync in the first place.

I believe all the ROMA servers are using the osmosis 
--read-change-interval task coupled to a --write-pgsql-change task.  If 
that's the case and data gets out of sync then I have a bug I need to 
know about.  The --read-change-interval task updates the timestamp as 
the very last thing it does.  Prior to doing that it performs all the 
updates to the pgsql database and all updates are performed within a 
single database transaction.  If it fails at any point the database 
transaction will rollback and the timestamp will not be updated.  If it 
somehow fails between committing the database and writing the timestamp 
then it will re-apply the changes when it next runs.  It should be very 
reliable, but if it's not I'd like to know.

I don't want to take you off topic though, an audit is a good idea.

80n wrote:
> On Mon, Dec 15, 2008 at 9:57 AM, Mathieu Arnold <mat at mat.cc> wrote:
>
>     +--On 15 décembre 2008 09:41:34 +0000 80n <80n80n at gmail.com
>     <mailto:80n80n at gmail.com>> wrote:
>     | Are you saying, very simply, a count of all the tags for nodes,
>     ways and
>     | relations for any given timestamp?
>     |
>     | For example:
>     | 2008-12-15 08:04 = 898266982
>     | 2008-12-15 08:05 = 898278312
>     | 2008-12:15 08:06 = 898279938
>     | etc
>     |
>     | I was thinking of something with a little bit more granularity,
>     such as
>     | the sum of the seconds component of each time stamp.  This would
>     also, I
>     | think, be quicker to recalculate if needed.
>     |
>     | If the checksum was stored in a table for every minute diff
>     update then,
>     | if something goes wrong, it would be easy to see when it went
>     wrong, and
>     | make it easier to correct by re-running diffs from that moment.
>
>     I don't know how much time it would take on other ROMA to select
>     count(*)
>     from nodes, but on mine, it's more than a few minutes, so, it
>     could not be
>     done every minute :-)
>
>
> Well the obvious way to do it is to maintain a count of insertions, 
> updates and deletions.  Scanning the whole table every time would be 
> silly.
>
>  
>
>
>     It could be done every hour or so though.
>
>     --
>     Mathieu Arnold
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>   


_______________________________________________
dev mailing list
dev at openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev





More information about the dev mailing list