<HTML><BODY>Hi all,<br><br>I agree on some points with Christoph, Peter and Martin, and on some points<br>with Jarek. So let me comment on the remaining ones, and suggest some<br> future course of action for myself and others in regard to this import, if I may.<br><br>First I want to point out that the vast majority of OSM data imports [2] deal with<br>single node-objects (e.g. bus stops, addresses etc.), linear objects (roads)<br>and sometimes man-made "small" closed polygons (buildings). There are however<br>few imports dealing with naturally-shaped polygons (land cover).<br>The last category of objects have some unique properties which create fascinating<br>challenges, in particular:<br><br>1. Natural objects have "blurred" and fractal borders, leading to endless disputes<br> on whose border drawing is right, when no one is right — it is just impossible<br> to represent such a border as a polygon with a pre-defined error margin.<br><br>2. Conflation with existing natural borders. In OSM, it is often expected that<br> two natural areas contacting each other have a single border. As it comes from<br> #1, two different humans/tools would always draw this shared border differently.<br> There should be a way to decide on a one border, however, see below.<br><br>3. Just sheer amount of data and need to control it. As it follows from #1,<br> machines are especially eager to generate polygons with crazy high amount of<br> nodes. It is then a human's task to filter them out leaving a reasonable<br> subset. Again, some people will argue that a particular forest area does not<br> look right when compared against a particular aerial imagery.<br> And it will never be, even if we allow it to have a crazy high number of<br> nodes. Then other people would argue that their editor freezes when<br> rendering such area.<br><br>Note how this is different with man-made features: their borders are always sharp.<br>People would be uncomfortable living in fuzzy buildings, or riding fractal roads.<br>Even our farmlands have straight borders, unless they meet a forest, where<br>natural "chaos" comes in conflict with human's desire to simpify things.<br><br>My understanding (please correct me if I am wrong!) is that OSM currently<br>lacks some good tools/checks to assist with areas.<br>There are tools such as "simplify way", "simplify area", "merge contours",<br>"merge areas" etc., and I used them countless times to "straighten out"<br>natural objects along borders.<br>There also are tools to conflate nodes and linear ways to help importing<br>such data [5]. However, these tools do not scale well or do not work for closed areas.<br>No warnings are often given for intersecting natural areas in editors such as<br>JOSM, possibly because of #1 above, even though I do wish to be reminded when<br>a forest slips into water.<br>This partially explains to me community's reluctance of accepting such imports:<br>we do not have tools to do quality assurance of such data, and doing it manually<br>is daunting. Can someone create a Summer of Code task to work on such tools,<br>or something similar?<br><br><br>Now, to some comments and answers.<br><br>> it should not be an acceptable excuse for (e.g.) topological errors<br>Agree, topological errors are not acceptable. By that I mean things such as<br>unclosed polygons, multipolygons with inner ways outside outer ways,<br>extremely long ways, self-intersecting ways etc.<br><br>> that the average mappers produce or use them as well.<br>Pray tell, what is an average mapper's work? Shouldn't we just refuse accepting<br>an average mapper's contributions until one has a diploma of a qualified<br>OSM-mapper :-) ?<br><br>I would very much gratefully accept an *actionable* critique in form:<br>"In your import, X is bad because Y. To make it better, do Z". There is some<br>good criticism in this thread, and in the talk-se thread, that we will turn<br>into action.<br><br>> Almost only new ways and it seems everything is tagged with landuse=forest,<br>> no matter if it's natural scrub or wetland or whatever else<br><br>This import does have areas of forest, scrub, wetland, and their combinations.<br>We also have additional tags for forests to clarify their type (broadleaved etc.),<br>available, but I omitted them for the time being to cut down on number of polygons.<br>One is always surprised to see how frequently types of forest change each other<br>in reality, and how this translates into increase of number of polygons.<br><br>>> The first 16 of those changesets<br>>> uploaded nodes, 10k each of them (i.e. total of 160000 nodes).<br>> Limitations of API should be dealt with for the convenience of the<br>> people doing the mapping, not of the API.<br><br>I used the JOSM's uploading interface to do this first medium sized import.<br>Just to learn how good it deals with larger changesets, network issues,<br>intermittent conflicts etc. Things went much better than I expected. At least it<br>did not screw the whole database when a minor conflict was discovered somewhere<br>in the middle of upload, and allowed me to fix it and continue from the<br>point of interruption.<br><br>I am aware about other tools that reorder changesets to look more human-like,<br>not just "nodes first, ways second, relations last" or whatever. I plan<br>to use these tools later for really large subareas of this import.<br><br>If JOSM's style of uploading of larger changes looks so disturbing, shouldn't it<br>just be fixed? Is there already a bug on it? Should I create such a bug?<br><br>>It might be charitable to note here that there is a large discussion<br>> about what is supposed to be the meaning of landuse=forest<br><br>The discussion of "landuse=forest" vs "natural=wood" vs "..." is endless and I<br>won't be able to add anything valuable to it. The majority of forests in Sweden<br>are tagged with "landuse=forest". When I used "natural=wood", I was asked not to.<br>I am a simple man, and I know it is a forest when I see it [1]. If I am unsure<br>for a certain area, I leave it untouched, no matter what the import data<br>layer says.<br><br>Now, to future improvements.<br><br>> But outlines also don't match. Strange, but ok. A bit more to the east<br>> I noticed the lakeshore: It intersects the forests multiple times.<br>> Strange. A bit to the south, the island Lövholmen intersects the forest<br>> yet again. And so on and so on.<br><br>This bothers me as well. I personally spent a lot of time to manually merge<br>closely placed but not matching and sometimes intersecting forest/water borders<br>of pre-existing objects, both drawn by humans or created by past land cover<br>imports such as CORINE [3]. We definitely do not want to create more of such<br>borders. I do have some ideas on how to detect and/or resolve at least some of<br>such conflation problems, and I am working on improving current conflation<br>script [4] to take even more details of pre-existing objects into account.<br><br>Currently the script already avoids to import or marks for human resolution<br>cases when land use of pre-existing data and new data possibly intersect.<br>So it was not a data/machine error, they did their job.<br>It was my personal overlook as I mostly focused on residential/forest<br>intersections.<br><br>We won't be trying to import data for further subareas of the country until<br>the intersection problems and some other annoying issues are reliably detected and<br>preferably automatically solved to reduce chances of human errors.<br>If there are other systematic problems present in this first batch that you<br>would like to be fixed (and in what manner), please inform us.<br><br>Thank you every one and sorry for long read.<br><br>[1] https://en.wikipedia.org/wiki/I_know_it_when_I_see_it<br>[2] https://wiki.openstreetmap.org/wiki/Import/Catalogue<br>[3] https://wiki.openstreetmap.org/wiki/WikiProject_Corine_Land_Cover<br>[4] https://github.com/grigory-rechistov/nmd-osm-tools/blob/master/conflate.py<br>[5] https://wiki.openstreetmap.org/wiki/Conflation<br><br><br><blockquote style="border-left:1px solid #0857A6; margin:10px; padding:0 0 0 10px;">
Среда, 24 апреля 2019, 2:39 +03:00 от Jarek Piórkowski <jarek@piorkowski.ca>:<br>
<br>
<div id="">
<div class="js-helper js-readmsg-msg">
<style type="text/css"></style>
<div>
<div id="style_15560627930000000649_BODY">On Tue, 23 Apr 2019 at 16:59, Peter Barth <<a href="mailto:osm-peda@won2.de">osm-peda@won2.de</a>> wrote:<br>
> I didn't read the plan btw, but wanted to read the ML if there really<br>
> was community acceptance as any note about this was left out in this<br>
> thread. A huge thread, all swedish, so no idea if swedish community is<br>
> ok or not. Did they accept or decline or abstain?!<br>
<br>
Hello Peter,<br>
<br>
This comment attracted my attention. In my experience it is quite<br>
usual that when local issues are discussed in local forums, it is done<br>
in the local language, whether English, French or Swedish. I have not<br>
often seen a summary written in a foreign language towards the end of<br>
the discussion. Perhaps it is common in the German-speaking forums or<br>
mailing lists?<br>
<br>
> The first 16 of those changesets<br>
> uploaded nodes, 10k each of them (i.e. total of 160000 nodes). No ways<br>
> or relations. Ok, so a huge change, split into changesets leaving<br>
> traceability to the mapper instead of the importer. As always, I want<br>
> to add. Changeset Nr. 17[2] actually adds something. Of course again<br>
> a quite large one but at least achavi[3] could load it.<br>
<br>
Limitations of API should be dealt with for the convenience of the<br>
people doing the mapping, not of the API. OSM tooling for viewing<br>
changesets and area history is known to be poor but this cannot be a<br>
determining factor for contributing to OSM (otherwise I request that<br>
rosemary 0.4.4 or wheelmap changesets with bounding boxes spanning the<br>
globe be blocked immediately). Would you rather have contributors<br>
figuring out how to deal with API limitations and making pretty<br>
changesets, or doing mapping or indeed importing?<br>
<br>
> Almost only new ways<br>
<br>
This cannot be surprising in an area where forests are largely unmapped.<br>
<br>
> and it seems everything is tagged with<br>
> landuse=forest, no matter if it's natural scrub or wetland or whatever<br>
> else. Seems wrong to me and taginfo[4].<br>
<br>
It might be charitable to note here that there is a large discussion<br>
about what is supposed to be the meaning of landuse=forest, and of<br>
landuse and landcover tags in general, with no OSM-wide consensus that<br>
I've seen so far. If there were trees grown in an area, they were cut<br>
down for wood, and it's regrowing as scrub, is the land still "being<br>
used as a forest"? Is wetland=swamp automatically not a forest?<br>
<br>
> I opened a small test area in an editor, a small island[5]. It has an<br>
> offset to all imageries out there. Ok, imagery can be wrong for sure.<br>
> But outlines also don't match. Strange, but ok. A bit more to the east<br>
> I noticed the lakeshore: It intersects the forests multiple times.<br>
> Strange. A bit to the south, the island Lövholmen intersects the forest<br>
> yet again. And so on and so on.<br>
<br>
Indeed from imagery it looks like the lake, which was previously<br>
mapped (seemingly by hand in changeset 29491171), is inaccurate, and<br>
the bits of forest that are now mapped at 16/59.0365/16.0570 appear<br>
fairly accurate to me comparing to local orthophoto (Lantmäteriet<br>
Historic Orthophoto 1975). From the achavi visualizations we can see<br>
that the lake outline was largely unchanged. Would you have liked to<br>
see the lakes manually fixed while importing forests?<br>
<br>
> From this quite small random sample I'd argue that this is a very low<br>
> quality import. I'm not really astonished about that, but I'm<br>
> questioning if it isn't time to increase our quality standards wrt<br>
> imports and introduce import permissions as opposed to just ignore<br>
> criticism and wait a week or two to import.<br>
<br>
And who might issue such a permission? The local community? The<br>
already overworked DWG? OSMF?<br>
<br>
Best,<br>
Jarek<br>
<br>
_______________________________________________<br>
Imports mailing list<br>
<a href="mailto:Imports@openstreetmap.org">Imports@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/imports" target="_blank">https://lists.openstreetmap.org/listinfo/imports</a><br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<br>С наилучшими пожеланиями,<br>Григорий Речистов.<br>Med vänliga hälsningar,<br>Grigory Rechistov<br>With best regards,<br>Grigory Rechistov<br></BODY></HTML>