[Imports] Importing addresses in New South Wales, Australia

michael spreng talk at osm.datendelphin.net
Sat Jul 7 11:18:58 UTC 2018


Hi

If the addresses just differ in house number you can merge them and put
both numbers in the house number tag, separated by comma: 4,6 for example.

Michael
On 07/07/18 13:09, Dion Moult wrote:
> Thanks Andrew, however there is one more problem I am running into. I
> have scoured through in a bit more detail and fixed that duplicate
> address you mentioned. I also found 2 more mistakes which I missed the
> first time around.
> 
> Now a basic thing I missed is that I didn't run JOSM's validate tool. If
> I run it, I find two issues: one is duplicate nodes (identical nodes on
> top of each other) and another is nodes at the same position (nodes with
> different addresses, but same location). Resolving the first type of
> validation error is easy: just delete the duplicate nodes so there is
> only one node. However, what should I do for the second error?
> 
> If I delete the nodes, I reduce the amount of useful data on the map. If
> I displace the nodes, I could introduce errors. If I leave the nodes, it
> may look "messy" on the map. What is the preferred strategy?
> 
> You can see the latest OSM file and run the validation yourself here:
> 
> https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm
> 
> Dion Moult
> 
> 
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On July 7, 2018 11:38 AM, Andrew Harvey <andrew.harvey4 at gmail.com> wrote:
> 
>> I think you missed https://www.openstreetmap.org/way/322975639
>>
>> Good idea with the JOSM mapstyle!
>>
>> You have the right intentions (your trying to avoid duplicates and not
>> just blindly dumping in data) so I'd say if you're keen then just go
>> ahead and do the importing, no need to ask here every time.
>>
>> On 6 July 2018 at 20:19, Dion Moult <dion at thinkmoult.com
>> <mailto:dion at thinkmoult.com>> wrote:
>>
>>     Thanks very much Andrew! I've made the fixes! I have also added a
>>     MapCSS mapstyle in JOSM so that I can very easily visually see any
>>     existing addr:housenumbers. You can see it here:
>>
>>     https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/src/elemstyles.mapcss
>>     <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/src/elemstyles.mapcss>
>>
>>     With this mapstyle, I discovered a few more buildings where I had
>>     accidentally doubled up on addresses! I've deleted them and here
>>     is the updated OSM file. I would really appreciate if you can do
>>     another double check. If everything is OK I will upload it to OSM
>>     and move to a different area to import addresses!
>>
>>     https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm
>>     <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm>
>>
>>     Dion Moult
>>
>>
>>     ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>     On July 6, 2018 6:19 PM, Andrew Harvey <andrew.harvey4 at gmail.com
>>     <mailto:andrew.harvey4 at gmail.com>> wrote:
>>
>>>     I took a look at your OSM file
>>>     https://www.openstreetmap.org/way/318055214
>>>     <https://www.openstreetmap.org/way/318055214> already has the
>>>     same address that you've included. While it's not harmful,
>>>     probably better to exclude adding duplicate data here. There are
>>>     a number of similar issues around the same area.
>>>
>>>     Then there's also https://www.openstreetmap.org/way/322975652
>>>     <https://www.openstreetmap.org/way/322975652>, which has
>>>     housenumber 3 but shows up as 61 in your proposed import.
>>>
>>>
>>>     On 1 July 2018 at 18:07, Dion Moult <dion at thinkmoult.com
>>>     <mailto:dion at thinkmoult.com>> wrote:
>>>
>>>         So I've begun with my first batch of address imports in
>>>         Epping, NSW. The area I focused on is shown in the picture below:
>>>
>>>         https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/img/epping-scope.png
>>>         <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/img/epping-scope.png>
>>>
>>>         It is the area bounded by Carlingford Road, Blaxland Road,
>>>         and the Epping suburb boundary. The screenshot is in JOSM
>>>         with the LPI NSW Base Map. I drew a way clicking everywhere I
>>>         saw an address in the LPI NSW Base Map. I made the following
>>>         mapping decisions:
>>>
>>>          1. If there is existing address information, I am not going
>>>         to overwrite it and so I don't add a node.
>>>          2. If there is an existing building on the lot but no
>>>         address information, I will add a node. However, I will not
>>>         merge the address tags onto the building. This will be a job
>>>         for another mapper.
>>>          3. I only add a node if it is absolutely unambiguous from
>>>         the LPI NSW Base Map that there is a clear property boundary
>>>         and a number shown that there is a unique address there to
>>>         the best ability of my human brain. If there is anything
>>>         uncertain, I don't add a node. The purpose of this is to get
>>>         99% of the way there, and minimise false positives.
>>>
>>>         I feel these decisions minimise the risk of the import and
>>>         keep the import simple. There should be no duplication of
>>>         data and no overwriting of data. After running the script, I
>>>         created this result:
>>>
>>>         https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/img/epping-results.png
>>>         <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/img/epping-results.png>
>>>
>>>         You can see that the address nodes very neatly line up and
>>>         are at the exact centroid of the property. The data set is
>>>         roughly 1600 addresses being imported. Along the way I
>>>         improved the script a bit to accomodate some errors from the
>>>         server, parallelize the requests a bit, and added some edge
>>>         cases which I noticed (A road named "The Boulevarde" will
>>>         return as a Null road type, even though "Boulevard" is listed
>>>         in their appendix). To process these 1600 addresses took
>>>         about half an hour of human time.
>>>
>>>         I've saved out the output to this link:
>>>
>>>         https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm
>>>         <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm>
>>>
>>>         Please download the .osm file and review it in JOSM. I have
>>>         not uploaded this to OSM as a changeset and will not until I
>>>         get another pair of eyeballs over it :) I look forward to any
>>>         comments and if there are any red flags! If everything goes
>>>         smoothly I will upload this, and do a larger area.
>>>
>>>         Here is the link to the repo for anybody who wants to
>>>         contribute:
>>>         https://gitlab.com/dionmoult/osm-nsw-address-import/
>>>         <https://gitlab.com/dionmoult/osm-nsw-address-import/>
>>>         (because Github is proprietary).
>>>
>>>         Dion Moult
>>>
>>>         ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>
>>>
>>>         On June 27, 2018 8:04 AM, Dion Moult <dion at thinkmoult.com
>>>         <mailto:dion at thinkmoult.com>> wrote:
>>>
>>>         > Thanks for the response Andrew! Absolutely, sometimes
>>>         government information is wrong, sometimes we represent it in
>>>         a better way, etc: this method allows the mapper to make a
>>>         judgement on a small case by case of exactly what to do.
>>>         >
>>>         > Probably something I didn't explicitly mention before, but
>>>         of course as I add address information I plan to do it slowly
>>>         and post it on talk-au for regular review and local QA.
>>>         >
>>>         > If there aren't any issues I will create an import plan on
>>>         the wiki this weekend, along with some data to review.
>>>         >
>>>         > Dion Moult
>>>         >
>>>         > -------- Original Message --------
>>>         >
>>>         > On 24 Jun 2018, 13:54, Andrew Harvey <
>>>         andrew.harvey4 at gmail.com <mailto:andrew.harvey4 at gmail.com>>
>>>         wrote:
>>>         >
>>>         > > Overall I'm okay with the approach you've outlined.
>>>         > >
>>>         > > > and then upload the changeset tagged as
>>>         `source:import=NSW LPI Web Services`, this will allow anybody
>>>         who is checking the history to know how this data was gotten.
>>>         > >
>>>         > > +1
>>>         > >
>>>         > > > There will be no automatic data overwriting or data
>>>         conflicts.
>>>         > >
>>>         > > That's good! I commented on the talk-au list, but to
>>>         expand on a few points, OSM is all about mapping what's on
>>>         the ground. There are many cases where the government
>>>         supplied address points database conflicts with information
>>>         on the ground. The on the ground scenario is what should
>>>         prevail in OSM, in my opinion. If anyone wants a government
>>>         supplied address points database, OpenAddresess does a great
>>>         job at collating that.
>>>         > >
>>>         > > Attached or at https://snag.gy/3jBX5a.jpg you can see a
>>>         place where I've mapped out an address different to the NSW
>>>         LPI Basemap, and different to GNAF, instead reflecting what's
>>>         on the ground.
>>>         > >
>>>         > > There's also differing views of how to map addresses in
>>>         OSM, eg. on an entrance node, on a building, on a residential
>>>         land parcel, as an unconnected address node. Additionally
>>>         addresses can be duplicated, eg. shops or offices with the
>>>         same address.
>>>         > >
>>>         > > My point is, I'm not against importing this data, but I
>>>         think we should pick and choose what makes sense to bring in,
>>>         and do it in a way that doesn't scare mappers away from
>>>         deleting or changing that imported address data when they
>>>         find it doesn't represent what's on the ground.
>>>         > >
>>>         > > On 23 June 2018 at 21:15, Dion Moult <dion at thinkmoult.com
>>>         <mailto:dion at thinkmoult.com>> wrote:
>>>         > >
>>>         > > > Good morning all!
>>>         > > >
>>>         > > > The state of New South Wales (where Sydney is) in
>>>         Australia currently has very minimal address information. I
>>>         am looking to speed up the mapping of addresses. These can be
>>>         simply nodes which have addr:street and addr:housenumber tags
>>>         at a bare minimum, but of course could be better things like
>>>         part of a building.
>>>         > > >
>>>         > > > Currently, apart from local knowledge, addresses can be
>>>         mapped using the LPI (Land and Property Information) Base Map
>>>         that we have explicit permission provided by the NSW
>>>         government to use.
>>>         > > >
>>>         > > >
>>>         https://wiki.openstreetmap.org/wiki/Contributors#Australia
>>>         <https://wiki.openstreetmap.org/wiki/Contributors#Australia>
>>>         > > >
>>>         > > >
>>>         https://wiki.openstreetmap.org/wiki/Attribution/New_South_Wales_Government_Data
>>>         <https://wiki.openstreetmap.org/wiki/Attribution/New_South_Wales_Government_Data>
>>>         > > >
>>>         > > > The map provides raster tiles of property boundaries,
>>>         along with the housenumber of the property. It is a little
>>>         ambiguous, however, at intersections which street the address
>>>         belongs to. A mapper is currently able to create a node,
>>>         interpret the base map background, and then tag it as necessary.
>>>         > > >
>>>         > > > However, we have explicit permission to use all of the
>>>         LPI web services, and the service API allows us to do two things:
>>>         > > >
>>>         > > > 1. Ask the question "At this coordinate, what is the
>>>         address?"
>>>         > > >
>>>         > > > 2. Ask the question "What is the coordinate of this
>>>         address?"
>>>         > > >
>>>         > > > Therefore, by roughly choosing coordinates (JOSM has
>>>         the Edit->Copy Coordinates feature) that we know is within
>>>         property boundaries (the boundary is shown in the LPI Base
>>>         Map in JOSM that we always have turned on anyway), we can use
>>>         a small script that will query the LPI Web Services API, ask
>>>         those two questions, and then automatically place nodes at
>>>         the accurate government centroid of the address. We can then
>>>         review the results manually, and then upload the changeset
>>>         tagged as `source:import=NSW LPI Web Services`, this will
>>>         allow anybody who is checking the history to know how this
>>>         data was gotten. This speeds up the adding and tagging of
>>>         nodes as the mapper does not need to manually type in data
>>>         (which is prone to spelling mistakes, and the housenumber in
>>>         the raster base map is not a high resolution and can be
>>>         misread), does not need to interpret ambiguous intersections
>>>         where street is not so clear, and will improve the quality of
>>>         the node placement as it will be at the actual centroid of
>>>         the property, not randomly to the mapper's liking.
>>>         > > >
>>>         > > > As you can see I am not proposing a huge data import,
>>>         so there is no actual data to review. Just a semi-automation
>>>         of the manual address tagging that we do now anyway. However
>>>         by some interpretation it could be called an "import" as it
>>>         is semi-automated node placement based off a government API
>>>         query. It is up to the individual mapper to manually choose
>>>         coordinates that they want to tag addresses at - so if the
>>>         mapper sees that addresses have already been tagged, they
>>>         will simply not query the LPI services for the address at
>>>         that point. There will be no automatic data overwriting or
>>>         data conflicts.
>>>         > > >
>>>         > > > I have discussed this on the talk-au mailing list, and
>>>         the responses seem positive. It's quite a conservative
>>>         proposal, after all. You can see the thread here:
>>>         > > >
>>>         > > >
>>>         https://lists.openstreetmap.org/pipermail/talk-au/2018-June/011937.html
>>>         <https://lists.openstreetmap.org/pipermail/talk-au/2018-June/011937.html>
>>>         > > >
>>>         > > > The small script along with sample data is shown here:
>>>         > > >
>>>         > > >
>>>         https://gist.github.com/Moult/5821c74fb792b7afa5d758aebea68e40 <https://gist.github.com/Moult/5821c74fb792b7afa5d758aebea68e40>
>>>         > > >
>>>         > > > I did a small test of 17 nodes in this changeset to
>>>         show how it would work. I manually reviewed it and I know
>>>         this area based off local knowledge.
>>>         > > >
>>>         > > > https://www.openstreetmap.org/changeset/59909707
>>>         <https://www.openstreetmap.org/changeset/59909707>
>>>         > > >
>>>         > > > Looking forward to your comments!
>>>         > > >
>>>         > > > Dion Moult
>>>         > > >
>>>         > > > _______________________________________________
>>>         > > >
>>>         > > > Imports mailing list
>>>         > > >
>>>         > > > Imports at openstreetmap.org
>>>         <mailto:Imports at openstreetmap.org>
>>>         > > >
>>>         > > > https://lists.openstreetmap.org/listinfo/imports
>>>         <https://lists.openstreetmap.org/listinfo/imports>
>>
> 
> 
> 
> _______________________________________________
> Imports mailing list
> Imports at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
> 




More information about the Imports mailing list