[Imports] (Estonia) Maa-amet building geometry update

fghj753 fghj753 at hot.ee
Sun May 30 21:10:06 UTC 2021


Hello. 

 

I was told in talk-ee that I should send single-paragraph notice to this
mailing list, but I think you deserve bit longer message. There's old and
long-lasting issue in Estonia with the 2008 building import by user
Verbatium. Primarily it had remarkably low quality when it was made and
awful quality by modern standards, but it also came from unknown source.
Over the past decade, local community has discussed the import several
times, but never reached definitive conclusion. For the past few months I
have been researching on the matter and after consulting with various sides
and analysing imported content, I've reached conclusion that buildings were
vectorised from Estonian Land Board's WMS service's Basic Map (1994-2007),
which was source allowed to be used in OSM. 

 

I'd like to run a geometry import in Harjumaa, Estonia for buildings where
all nodes were last modified at least 10 years ago. As you might know,
geometry in OSM is defined via nodes and ways get their shape via links to
nodes. Source dataset is Maa-amet's Estonian topographic dataset (ETAK),
which has already been used for OSM building imports since 2019. Main focus
and around 80% of modified buildings would be remains of Verbatium's import.
Conflation is achieved by only attempting to update buildings where no nodes
are shared with other ways, OSM version has less nodes than reference
dataset and there are no tags on any of way's nodes. OSM and source's
buildings are matched by comparing bounding boxes overlap - building is
updated only if there's exactly 1 match with at least 15% overlap.

 

Wiki page for import is
https://wiki.openstreetmap.org/wiki/Maa-amet_building_geometry_update and
account to be used is fghj753_import. Upload process to update 32000 out of
44000 buildings took on a testing VM around 10 hours. I should inform you
that script uses not just writing, but also reading operations of API. Per
every processed building WayFull and per every node NodeWays API functions
are called. Thanks to OSM's new Prometheus server it looks like the best
time to start import is around 7 PM UTC.

 

I have few finalizing questions. Should "source:*=Maa-amet 2021" tag be
duplicated onto elements (buildings) and if so, under which subkey
(source:geometry?), or would "source=Maa-amet 2021" on changeset suffice?
Previous address imports have added source:addr to each building and also
regular source tag on changeset. 

Secondly import guidelines mentioned something about spreading import across
wider timeframe, citing old Tiger imports. How important is that and how to
achieve that? When i was running import on local VM, it averaged around 3000
buildings per hour, comparable pace as previous 2019 imports. 2008 import
ran for over 25 hours straight, importing about 2000 buildings per hour.
Should I maybe try to get it running on Raspberry, to throttle import by
using Pi's slow hardware?

Are there some recommendations on how to improve import documentation? I
wasn't sure what exactly to write into import documentation as wiki
documentation of all previous Estonian imports combined have roughly length
of this email. Currently I haven't added row to Import/Catalogue yet. Should
I add line for 2008 import as well? 

 

Best regards,

Fghj753

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20210531/111f5917/attachment.htm>


More information about the Imports mailing list