[josm-dev] Problem with referrers

[Karl Guggisberg]

Hi there,

> jiri wrote:
> I'm not talking about history, I've just mentioned few cases where JOSM
use copies of primitives.
I think, framm got confused.

> framm wrote:
> you're trying to keep a "breadcrumb" trail of  what happened to an object,
I hope not! (although, this is the way the undo/redo mechanism is
implemented today, sort of).

> jiri wrote:
> Prototypes will be probably used when parsing xml from API (instead of
similar OsmData class that's already there). 
> Then when you copy a node
> - properties of the Node will be saved to the NodePrototype and later new
Node will be created based on that values.
Yes, like today there's an internal class used in the OSM parser. It plays a
role similar to that of the Prototype classes. 

> jiri wrote: 
> Prototype will be generated only when copy of some primitive is needed. So
for example to allow undo in some 
> actions or for Copy&Paste action.
Exactly, and just for the sake of naming suggestions: in this context they
could be called NodeMemento, WayMemento, etc.
(really just for the sake of examples, I'm not in favor of these names, they
would be used in the context of undo/redo, but not in the general context of
OSM primitives.).
ImmutableNode, ImmutableWay? Not really, I think Jiri's prototypes are not
meant to be immutable. 
In some software architectures there's also the concept of "value objects"
(what's is sent from a remote client to a server) and "value objects" are
sometimes called "detached" (from their "entity objects"). So would
DetachedNode, DetachedWay, ... be an alternative? 

Today, whenever you need to walk from children to parents (from nodes to
their parent ways, from members to their parent relations), you have to loop
over all primitives in order to find the suitable parents. If you know that
you will have a lot of child->parent relations to traverse you can build up
the respective data structures and reuse them instead of looping over
primitives. That's what BackreferenceDataSet and
CollectingBackreferenceVisitor are used for. 

In the future this won't be necessary any more. Traversing parent->child
will be as efficient as traversing child->parent. 
Child->parent relations will be maintained on the fly in the data structure
we today know as DataSet.

> framm worte:
> Is the "traditional" setup not something like this: You have a model class
which keeps the data, and then you have 
> a controller class which performs all sorts of stuff on the models?
I hope that no OO evangelist are reading this list because otherwise we are
going to end up in a flame war about wheter and to what extend data and
behaviour should be separated or not. I'm not talking about MVC here, OO
evangelist would love that, but some (like Folwer) may call this separation
of "data" and a controller which does all sort of stuff with the data an
"anaemic" approach.

> framm wrote
> Another thing that may be worth considering at this time is whether you
might want to separate out the display stuff. 
I've already started to do that, see getDisplayName(...).  We had some
discussions about this in a trac ticket,
   http://josm.openstreetmap.de/ticket/3261

-- Karl 


-----Ursprüngliche Nachricht-----
Von: josm-dev-bounces at openstreetmap.org
[mailto:josm-dev-bounces at openstreetmap.org] Im Auftrag von Jiri Klement
Gesendet: Montag, 5. Oktober 2009 19:29
An: Frederik Ramm
Cc: josm-dev
Betreff: Re: [josm-dev] Problem with referrers

On Mon, Oct 5, 2009 at 9:43 AM, Frederik Ramm <frederik at remote.org> wrote:
> Can you explain (or point me to a previous post that explains) what 
> referrers are good for?

For example when way has list of nodes then every node has that way as a
referrer. Also every primitive included in relation has that relation saved
as referrer. Referrers are needed in lots of places in JOSM code, for
example in DrawAction and DeleteCommand. It's obviously possible to get list
of referrers by iterating over dataset but that's too slow for big datasets.

>> Currently JOSM quite often makes copies of primitive (using for 
>> example cloneFrom or special constructors). It's not obvious whether 
>> the copy is added to the dataset or is kept in case we need to get 
>> back to old version of primitive (for example Command.cloneMap). 
>> That's a bit problem because I don't want to have such copy in 
>> referrers list, I want only primitives that are really used in an 
>> dataset.
>
> I'm trying to make sense of this (but might go off in the completely 
> wrong direction here) - you're trying to keep a "breadcrumb" trail of 
> what happened to an object, is that right? And you want referrer lists 
> to model that "family tree"?
>
> But why can the list not simply reside on the other side, i.e. in the 
> descendant object pointing back in history? I have the feeling that 
> you're duplicating information by distinguishing objects that are 
> "really used in a dataset" from those that aren't - should not the 
> dataset itself be the ultimate arbiter on what is "really used" and what
not?

I'm not talking about history, I've just mentioned few cases where JOSM use
copies of primitives.

OsmPrimitive is no longer just a storage for osm data. It has (will
have) methods that will automatically update referrers, spatial index and
other things. We need another class that will just hold data - that's what
PrimitivePrototype is for.

Let me explain it on example with DeleteCommand. Currently JOSM makes a copy
of deleted object so it can revert the change. Let's say we delete way
without it's nodes. Now I have now idea whether the copy of deleted way
should be listed as referrer to the nodes or not.

>> To workaround this issues I'm thinking about introducing new classes 
>> - PrimitivePrototype, NodePrototype, WayPrototype and RelationPrototype.
>> These classes will hold informations for one primitive, but it will 
>> not be possible to add these objects to the dataset. That will allow 
>> to clearly distinguish what's real primitive and whats just an backup 
>> copy for for example Command.cloneMap or Main.pasteBuffer.
>
> [...]
>
>
> Could one perhaps forgo the prototype objects you are suggesting and 
> instead create a "backup copy dataset" (that is not associated with a 
> visible layer); then postulate that every primitive always has to be a 
> member of exactly one dataset - either a "proper" dataset or the "backup
copy dataset"
> (or maybe others like it)?

Having "backup copy dataset" that will discard all event received from it's
primitives is another possibility. But I think it's cleaner to have
separated classes - one as simple data holder and another one as part of
real dataset with EastNorth coordinates, MapPaint fields etc.

> I am not entirely happy with making primitive constructors protected. 
> Is it so unthinkable that people, e.g. those writing plugins, would 
> like to use primitives for purposes that are unexpected for us - 
> purposes where they are not at all interested in things like "reliable 
> events"? Just because we need that somewhere in JOSM doesn't mean we 
> have to force it on everybody, or force them to write their own classes
instead of re-using ours.

If it turns out that somebody needs to do something unusual with primitives
then it's quite easy to provide special Dataset, that will miss that
features.

> feeling that it would be nice if our Node class was primarily that - 
> an OSM node - with perhaps some added JOSM bells and whistles. Maybe, 
> in a way, you were aiming at making your "node prototype" represent a 
> basic node, and the Node class would play the role of linking that 
> basic node into the greater JOSM context? But if that is right, then I 
> guess we should change the naming
> - i.e. instead of saying "datasets consist of primitives which are 
> made from prototypes" we should say "datasets consist of adapter 
> classes (or linkage classes, or instance classes, or context classes, 
> or wrapper classes or
> something) which refer to a primtive". I.e. instead of further and 
> further mangling the primitive and adding a prototype to its side, 
> make the primitive into a real primitive (which you called a 
> prototype) and but all the fancy stuff in the new PrimitiveContext class.

I think almost everywhere in JOSM are primitives used as PrimitiveContext
class. Most places in JOSM expect to update (future) spacial index when Node
coordinates are changed. So I think it's a bit late for changing names. But
otherwise I agree that it would be nice if Node was really just osm node
without extra features.

Now I should emphasis that PrimitivePrototypes are not meant like some data
holding class below OsmPrimitive. Prototypes are temporary objects used when
you need to make copy of OsmPrimitive. Maybe the name is not the best, any
other suggestions?


> --
>
> When would your prototype objects be created - only on demand or would 
> they always be there? Say I load a data layer from a file - does that 
> then create a set of primitives and an associated set of prototypes 
> (memory issue?), or would the prototype only be generated when required?

Prototype will be generated only when copy of some primitive is needed. So
for example to allow undo in some actions or for Copy&Paste action.

> How would you manage
> object identity - could you quickly sketch, for the following sequence 
>of  actions, at what time what prototypes and primitives would be 
>generated and  what OSM ID would be associated with them?
>
> * start JOSM
> * load data layer from API
> * create new, empty layer
> * copy node from data layer
> * insert node into new layer
> * upload new layer to API

Prototypes will be probably used when parsing xml from API (instead of
similar OsmData class that's already there). Then when you copy a node
- properties of the Node will be saved to the NodePrototype and later new
Node will be created based on that values.

> Bye
> Frederik
>
--
Jiri

_______________________________________________
josm-dev mailing list
josm-dev at openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev