[OSM-dev] Minute Diffs Broken
ssinger_pg at sympatico.ca
Tue May 5 04:47:06 BST 2009
On Tue, 5 May 2009, Brett Henderson wrote:
> That does look interesting. I'd hope to use that outside the main database
> though. My thoughts were to use triggers to populate short term flag tables
> which a single threaded process would read, use as keys to select modified
> data into an offline database, then clear. This offline database could then
> use a queueing system such as PgQ (I haven't seen it before, will have to
> check it out) to send events to the various consumers of the data. I'd like
> to minimise access to the central database if possible because 1. it will
> scale better, and 2. it adds less burden to existing DBAs.
I agree you'd only want one process pulling data from the central database
and then let other clients pull from another machine. You'd have to examine
how different your trigger + scanning process code will be from using PgQ
with 1 consumer that then stores the data in another db for publishing. You
should at least look to see what problems they solved.
One concern with trigger based systems is that for each real INSERT your
doing a second insert into a queue or journal table, but there might not be
a way around that.
More information about the dev