[Imports] (Estonia) Maa-amet building geometry update

fghj753 fghj753 at hot.ee
Tue Jun 1 11:19:07 UTC 2021


I’d like to run two changesets with ~200 buildings on live server to see how upload runs on other servers and so you could review first edits. I’ve added 60s delay after every changeset (40x) and 0.4s delay after every modified building (there are ~32k buildings), creating total delay of 04:13 h, in addition to projected 10h upload time.

Remember that these buildings are in rural area, hence the small changesets. When script reaches urban areas, changesets will be much larger.

May I continue to upload?

 

From: fghj753 <fghj753 at hot.ee> 
Sent: Monday, May 31, 2021 3:14 PM
To: imports at openstreetmap.org
Subject: Re: [Imports] (Estonia) Maa-amet building geometry update

 

> 

> I really, really, really dislike such debris on objects. Especially as it is often stale anyway - people

> updating building shape based on aerial/survey rarely remove such tags.

> 

> Though please ignore me if local community has a different preference. 

> 

> It is also a personal preference that is (I think) well motivated, not OSM rule.

> 

Only reply from local mailing list agrees with you. I also sent clarifying question to Maa-amet / Land Board, but I don’t think they will reply anytime soon. 

> 

> For technical side: interrupt script after it made N edits, then restart it the next day?

> Or sleep for N seconds after each edit?

> 

> When I run bot edit it typically sleeps one minute after every changeset

> 

Uploading changeset of 10000 changes took around 20 min on local VM. I’m afraid delaying just 1 minute between sets is not enough. For comparison, your bot made 100-300 changes per changeset in less than 2 minutes and then waited for 1 min. Waiting 0.5-1 sec per building (not node change) may double the time consumed, but seems to be more fair solution to delay edits. Also in that case uploading would start approaching full day, allowing to start import any time.  Honestly I’d prefer to run import overnight, but then leaving 2-4 hour chunk for next day doesn’t make much sense. The python script does support splitting the upload.

My personal objective on the matter is to complete import by Friday morning, but I probably don’t have time on Thursday evening to start it, meaning that import should be scheduled on Wednesay evening.

> Though I think that adding LIDAR based height data should be mentioned in summary posted here

Not much to comment here other than posting direct link to table:  <https://wiki.openstreetmap.org/wiki/Maa-amet_building_geometry_update#Tagging_Plans> https://wiki.openstreetmap.org/wiki/Maa-amet_building_geometry_update#Tagging_Plans and pasting table cell in question:

> Dataset also contains lidar-measured building height data for some buildings. Height is only added if any nodes of the building were modified, height=* is not present yet and height recorded in dataset is at least 3 m.

There were concerns about StreetComplete not asking building:levels quest for buildings with height data, but Mateusz has already opened PR on SC’s repo to solve the problem from the other end. Height data is stored as integer, meaning 1m precision. Also I’m planning to remove bit under 200 instances of undocumented (CityIdx), legacy (created_by) and duplicate tags (name == addr:housenumber) on buildings, where geometry is updated too. Planning to add height data and previous local building import’s signature key maaamet:ETAK.

I’ve added two rows to Import/Catalogue, but I also took liberty to unify date formatting of these 3 tables. That raised an odd question: How many days were in November 2009 (looking at „Spanish administrative borders“ Import)?

Fghj753

From: Mateusz Konieczny via Imports <imports at openstreetmap.org <mailto:imports at openstreetmap.org> >
Sent: Monday, May 31, 2021 11:20 AM
Cc: Imports <imports at openstreetmap <mailto:imports at openstreetmap.org> .org>
Subject: Re: [Imports] (Estonia) Maa-amet building geometry update

May 30, 2021, 23:10 by fghj753 at hot.ee <mailto:fghj753 at hot.ee> :

I have few finalizing questions. Should “source:*=Maa-amet 2021” tag be duplicated onto elements (buildings) and if so, under which subkey (source:geometry?), or would “source=Maa-amet 2021” on changeset suffice? Previous address imports have added source:addr to each building and also regular source tag on changeset.

I really, really, really dislike such debris on objects. Especially as it is often stale anyway - people

updating building shape based on aerial/survey rarely remove such tags.

Though please ignore me if local community has a different preference. 

It is also a personal preference that is (I think) well motivated, not OSM rule.

Secondly import guidelines mentioned something about spreading import across wider timeframe, citing old Tiger imports. How important is that and how to achieve that?

I suspect that reason for that is that it is easier to spot runaway script damaging data?

For technical side: interrupt script after it made N edits, then restart it the next day?

Or sleep for N seconds after each edit?

When I run bot edit it typically sleeps one minute after every changeset

(that are grouped into groups of close edits).

Are there some recommendations on how to improve import documentation?

https://wiki.openstreetmap.org/wiki/Maa-amet_building_geometry_update seems OK

Though I think that adding LIDAR based height data should be mentioned in summary

posted here

I wasn't sure what exactly to write into import documentation as wiki documentation of all previous Estonian imports combined have roughly length of this email. Currently I haven’t added row to Import/Catalogue yet. 

Should I add line for 2008 import as well?

Seems to be a good idea.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20210601/73150cd0/attachment.htm>


More information about the Imports mailing list