[Talk-ca] Did I just find a potentially significant bug in JOSM?

Dan Charrois dan at syz.com
Tue Mar 8 05:26:45 GMT 2011


Thanks Adam, and everyone else who responded to my message.  I'm glad to learn the bug is repeatable by others - that means I'm not going completely crazy.  Plus, it's nice to have a good understanding now why some of the ways I'd uploaded had mysteriously disappeared.

I'm glad you were able to distill down the problem so succinctly.  validatorfail.osm shows off the bug well, and is much easier to follow what's going on.

I'd apparently been lulled into trusting JOSM's ability to fix simple things like duplicate nodes a little too much.  It's definitely a time-saver when it works, but not if roads are lost in the process.  I'm hoping that a slightly revised procedure of fixing things just one category at a time (rather than at the top-level "Warnings" category) should be a good workaround.

In the meantime, I agree that a bug report should definitely be filed about this.  At the very least, disabling top-level "fixes" would force people to go through category by category.  I'm sure I'm not the only one who has trusted (or will trust) that the top-level "fix" is a good way to reduce the manual workload.  Your validatorfail.osm file is a perfect example of how to reproduce the problem, and should probably be included with the bug report - I can definitely try to figure out how to submit the bug report to http://josm.openstreetmap.de/ if you're busy or no one else has yet, and you don't mind my including the file.  But as the person who really narrowed down the issue, the "honor" for reporting it (if there is such a thing) should be yours.. :-)

In any case, those "blanket fixes" will be a thing of the past for me - I'm hoping that the category-by-category approach doesn't cause any issues.

Other than this hiccup, I've found JOSM to be quite intuitive and reliable, though as was mentioned, it is a work in progress and likely always will be.  It's just a matter of finding where its specific weaknesses are, and then avoiding those.

Dan

On 2011-Mar-07, at 8:31 PM, Adam Dunn wrote:

> Just reproduced this issue with JOSM 3751. Most warnings are not
> auto-fixable in Validator. I have only been able to find two "warn"
> level issues that validator will auto-fix: Nodes at same position, and
> duplicated nodes.
> 
> As stated by Dan, clicking on a sublevel within Warn (such as unnamed
> highway) will disable auto-fix, but selecting top level Warn will
> enable the auto-fix button (if there is a duplicate node sublevel).
> Using auto-fix when there are both duplicate nodes and unnamed
> highways will cause the dupe node *and* the unnamed highway to be
> deleted.
> 
> Method to reproduce (in 3751):
> 1 - Initialize a blank layer in JOSM (eg. download middle of ocean or
> open an empty .osm file).
> 2 - Create a way that is of highway={residential,secondary,etc
> (whatever is necessary to raise a "unnamed highway warning", not
> road)}
> 3 - Create a node (no tags necessary)
> 4 - Copy and paste node into same position (zoom in excessively to
> position duplicated node on top of original)
> 5 - Run Validator
> 6 - Click on top-level "Warnings" category, then click on the "Fix" button
> 7 - Watch road disappear
> 
> Alternative method:
> 1 - Download attached .osm file (this is a minimal file to reproduce
> the bug, and is easily human readable in any text editor)
> 2 - Run Validator
> 3 - Click on "Warnings" category, then click "Fix" button
> 4 - Watch road disappear
> 
> Adam
> 
> 
>> -----Original Message-----
>> From: Samuel Longiaru <longiaru at shaw.ca>
>> To: Dan Charrois <dan at syz.com>
>> Cc: Talk-CA OpenStreetMap <talk-ca at openstreetmap.org>
>> Subject: Re: [Talk-ca] Did I just find a potentially significant bug in
>> JOSM?
>> Date: Mon, 07 Mar 2011 18:13:37 -0800
>> 
>> Dan...
>> 
>> Have to admit that I VERY rarely if ever use the fix button, and then only
>> after I've isolated the issue for a specific error or warning.  I do errors
>> first, then warnings.  But after each correction, I revalidate to refresh
>> the list. Maybe that's overkill, but it keeps the exceptions from being
>> thrown.  I think you're getting the exceptions because you've modified the
>> data, but have not updated the list.  Usually, I just highlight the specific
>> error or warning, hit the SELECT button, then the "3" key to go there.  I
>> fix it manually if I can, then re-validate.  Then move to the next issue.
>> I'm too much of a scaredy-cat to highlight a whole class of errors and let
>> JOSM fix them automatically.  Like I said, I'm not sure JOSM likes me.
>> 
>> I'll try selecting an unnamed way warning and "fixing" to see what happens.
>> If it removes the way.... well, that's certainly ONE way to fix it.
>> 
>> Sam L.
>> 
>> 
>> -----Original Message-----
>> From: Dan Charrois <dan at syz.com>
>> To: Talk-CA OpenStreetMap <talk-ca at openstreetmap.org>
>> Subject: [Talk-ca] Did I just find a potentially significant bug in JOSM?
>> Date: Mon, 07 Mar 2011 17:42:09 -0700
>> 
>> I've spent the past several hours trying to find out out why some of my
>> edits resulted in some accidental road removals.  I think I've found the
>> problem, which seems to relate to an apparent bug in JOSM, or at least a
>> preference somewhere set to something unintuitive that I haven't been able
>> to find - as such, I thought it would be best to communicate with the group
>> to see if anyone's experienced this problem before, or at least give a heads
>> up as to what has been happening to me in case it's inadvertently happening
>> to others.
>> 
>> I downloaded an area around the Beaumont, Alberta region to try and add some
>> roads from Canvec that don't exist in OSM (roads which I could have sworn
>> I've added when I worked on the same area a month or two ago).  I then saved
>> this downloaded area, along with the changes I made, to an .osm file in
>> JOSM.  I ran the Validator which found a few duplicate nodes and such.
>> Telling it to fix errors, it did its thing.  And then, I clicked on
>> "warnings" and told it to fix those.  Lo and behold, some (though not all)
>> of the roads I'd just added suddenly disappeared.  Since validation was the
>> last step in my process before I uploaded the data, and I was often zoomed
>> all the way out when I did so, I hadn't noticed this happen before, but it
>> probably had been going on right before I uploaded my changes.  In
>> particular, sometimes I would replace lower quality OSM roads with better
>> Canvec data, and if JOSM was deleting some of those Canvec roads I'd added
>> in its "fix warnings" validation step,
>> those original OSM roads would have disappeared without replacements.
>> 
>> I tried to isolate further exactly what it was doing, and at least in this
>> situation it looks like it may be related to unnamed ways.  I have 41
>> unnamed ways in my data.  If I click on the "unnamed ways" folder
>> specifically in the "warnings" validation dialog, it doesn't give me the
>> option to fix them - it shouldn't, since it doesn't know what to call them
>> anyway.  But if I click on the parent (root) icon just labelled "Warnings",
>> the fix button is enabled, and I thought it was just supposed to fix all the
>> enclosed warnings in categories that it is able to fix.  When I click to
>> "fix" all warnings like this, in the progress bar, I see that it spends some
>> time "fixing" unnamed ways.  And then those ways are gone when it's done.
>> 
>> I was using JOSM 3751, so I checked for updates and found that 3961 was
>> available.  I've updated my JOSM to see if it's still an issue.  But now,
>> when I do the "global" fix warnings on the data set, it gets part way
>> through, then consistently gives me an unexpected exception (coding error).
>> When I dismiss that dialog, I find that those roads have again been removed,
>> so it seems as though this is still a problem.  The unexpected exception
>> isn't encouraging either.
>> 
>> I suppose that I can work around the problem by selectively choosing "fix"
>> in the validations warning area for each of the separate categories, instead
>> of doing them all at once.  Though now I'm unsure as to whether or not there
>> is still an underlying problem with fixing warnings that may give rise to an
>> inadvertent loss of data.  At the very least, I wanted to make a posting
>> about it so that others can be warned away from having the same problems I
>> have run into.
>> 
>> If anyone is interested in trying to duplicate the problems I'm having, just
>> let me know your email address and I can send you the .osm file I'm working
>> with (compressed, it's about 600 kB).  And of course, if anyone has any
>> ideas, or suggestions about something stupid I'm doing, please let me know!
>> 
>> Dan
>> --
>> Syzygy Research & Technology
>> Box 83, Legal, AB  T0G 1L0 Canada
>> Phone: 780-961-2213
>> 
>> 
>> _______________________________________________
>> Talk-ca mailing list
>> Talk-ca at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/talk-ca
>> 
>> _______________________________________________
>> Talk-ca mailing list
>> Talk-ca at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/talk-ca
>> 
>> _______________________________________________
>> Talk-ca mailing list
>> Talk-ca at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/talk-ca
>> 
>> _______________________________________________
>> Talk-ca mailing list
>> Talk-ca at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/talk-ca
>> 
>> 
> <validatorfail.osm>_______________________________________________
> Talk-ca mailing list
> Talk-ca at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk-ca

--
Syzygy Research & Technology
Box 83, Legal, AB  T0G 1L0 Canada
Phone: 780-961-2213




More information about the Talk-ca mailing list