Activity stream - extending Rails app data model / database

[Paweł Paprota]

Hi Frederik,

Thanks for the response!

On Sun, Jul 15, 2012, at 21:56, Frederik Ramm wrote:
> 
> I don't think that having activity objects is a good idea because that 
> would require modifying each process that generates them; e.g. every 
> time someone writes a blog entry you'd generate an activity record, 
> every time someone joins, etc.etc.
> 

That is true. On the other hand, so far I have identified only a small
number of them - new changeset is created, new user is created, new
diary is created. It is not a lot of work to write business logic for
that. Data redundancy is of course a valid concern.

> Generating activities on the fly seems the better approach to me, but 
> that of course requires more resources and we wouldn't like every user 
> to have an RSS feed that is queried every five minutes - or if we had, 
> we'd have to run that on a database mirror and not on the main database.
> 

I have started working with this solution because that's actually what
ajturner implemented - Changeset and DiaryEntry are used as activities.
The problem is that it got very inefficient very quickly. For example I
have a code like this:

def recent_activities(limit = 5)
    nearby_changesets = []
    self.nearby.each {|user| nearby_changesets +=
    user.recent_changesets.to_a}
    (self.recent_changesets + self.diary_entries +
    nearby_changesets).sort {|a,b| b.created_at <=> a.created_at}
  end

Now this generates a lot of SQL traffic. Of course this can be rewritten
to use one (or - constant) number of SQL statements but they will be
complex - so resources are being used either way.

> Ideally, I would like to see everything that an "activity stream server" 
> needs exported through something like our database diffs, so that the 
> load on the database server is minimal, and a *different* server can 
> pick up all these pieces and combine them into an activity picture.
> 

If this kind of architecture could be accommodated by admins/deployment
team then it is of course a no-brainer. I will look at activity stream
from this perspective and try to design accordingly.

Thanks again for your response. I understand this is non-essential when
compared to the core OSM functionality but still I believe it will add a
lot of value to the project, foster collaboration etc. And it is good to
see that people are not rejecting the idea just because it is
non-essential - I have had nice discussions on IRC about this already
and I really feel encouraged.

Paweł