[Imports] Importing addresses in New South Wales, Australia
michael spreng
talk at osm.datendelphin.net
Sat Jul 7 11:18:58 UTC 2018
Hi
If the addresses just differ in house number you can merge them and put
both numbers in the house number tag, separated by comma: 4,6 for example.
Michael
On 07/07/18 13:09, Dion Moult wrote:
> Thanks Andrew, however there is one more problem I am running into. I
> have scoured through in a bit more detail and fixed that duplicate
> address you mentioned. I also found 2 more mistakes which I missed the
> first time around.
>
> Now a basic thing I missed is that I didn't run JOSM's validate tool. If
> I run it, I find two issues: one is duplicate nodes (identical nodes on
> top of each other) and another is nodes at the same position (nodes with
> different addresses, but same location). Resolving the first type of
> validation error is easy: just delete the duplicate nodes so there is
> only one node. However, what should I do for the second error?
>
> If I delete the nodes, I reduce the amount of useful data on the map. If
> I displace the nodes, I could introduce errors. If I leave the nodes, it
> may look "messy" on the map. What is the preferred strategy?
>
> You can see the latest OSM file and run the validation yourself here:
>
> https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm
>
> Dion Moult
>
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On July 7, 2018 11:38 AM, Andrew Harvey <andrew.harvey4 at gmail.com> wrote:
>
>> I think you missed https://www.openstreetmap.org/way/322975639
>>
>> Good idea with the JOSM mapstyle!
>>
>> You have the right intentions (your trying to avoid duplicates and not
>> just blindly dumping in data) so I'd say if you're keen then just go
>> ahead and do the importing, no need to ask here every time.
>>
>> On 6 July 2018 at 20:19, Dion Moult <dion at thinkmoult.com
>> <mailto:dion at thinkmoult.com>> wrote:
>>
>> Thanks very much Andrew! I've made the fixes! I have also added a
>> MapCSS mapstyle in JOSM so that I can very easily visually see any
>> existing addr:housenumbers. You can see it here:
>>
>> https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/src/elemstyles.mapcss
>> <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/src/elemstyles.mapcss>
>>
>> With this mapstyle, I discovered a few more buildings where I had
>> accidentally doubled up on addresses! I've deleted them and here
>> is the updated OSM file. I would really appreciate if you can do
>> another double check. If everything is OK I will upload it to OSM
>> and move to a different area to import addresses!
>>
>> https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm
>> <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm>
>>
>> Dion Moult
>>
>>
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On July 6, 2018 6:19 PM, Andrew Harvey <andrew.harvey4 at gmail.com
>> <mailto:andrew.harvey4 at gmail.com>> wrote:
>>
>>> I took a look at your OSM file
>>> https://www.openstreetmap.org/way/318055214
>>> <https://www.openstreetmap.org/way/318055214> already has the
>>> same address that you've included. While it's not harmful,
>>> probably better to exclude adding duplicate data here. There are
>>> a number of similar issues around the same area.
>>>
>>> Then there's also https://www.openstreetmap.org/way/322975652
>>> <https://www.openstreetmap.org/way/322975652>, which has
>>> housenumber 3 but shows up as 61 in your proposed import.
>>>
>>>
>>> On 1 July 2018 at 18:07, Dion Moult <dion at thinkmoult.com
>>> <mailto:dion at thinkmoult.com>> wrote:
>>>
>>> So I've begun with my first batch of address imports in
>>> Epping, NSW. The area I focused on is shown in the picture below:
>>>
>>> https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/img/epping-scope.png
>>> <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/img/epping-scope.png>
>>>
>>> It is the area bounded by Carlingford Road, Blaxland Road,
>>> and the Epping suburb boundary. The screenshot is in JOSM
>>> with the LPI NSW Base Map. I drew a way clicking everywhere I
>>> saw an address in the LPI NSW Base Map. I made the following
>>> mapping decisions:
>>>
>>> 1. If there is existing address information, I am not going
>>> to overwrite it and so I don't add a node.
>>> 2. If there is an existing building on the lot but no
>>> address information, I will add a node. However, I will not
>>> merge the address tags onto the building. This will be a job
>>> for another mapper.
>>> 3. I only add a node if it is absolutely unambiguous from
>>> the LPI NSW Base Map that there is a clear property boundary
>>> and a number shown that there is a unique address there to
>>> the best ability of my human brain. If there is anything
>>> uncertain, I don't add a node. The purpose of this is to get
>>> 99% of the way there, and minimise false positives.
>>>
>>> I feel these decisions minimise the risk of the import and
>>> keep the import simple. There should be no duplication of
>>> data and no overwriting of data. After running the script, I
>>> created this result:
>>>
>>> https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/img/epping-results.png
>>> <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/img/epping-results.png>
>>>
>>> You can see that the address nodes very neatly line up and
>>> are at the exact centroid of the property. The data set is
>>> roughly 1600 addresses being imported. Along the way I
>>> improved the script a bit to accomodate some errors from the
>>> server, parallelize the requests a bit, and added some edge
>>> cases which I noticed (A road named "The Boulevarde" will
>>> return as a Null road type, even though "Boulevard" is listed
>>> in their appendix). To process these 1600 addresses took
>>> about half an hour of human time.
>>>
>>> I've saved out the output to this link:
>>>
>>> https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm
>>> <https://gitlab.com/dionmoult/osm-nsw-address-import/blob/master/review/EPPING-1.osm>
>>>
>>> Please download the .osm file and review it in JOSM. I have
>>> not uploaded this to OSM as a changeset and will not until I
>>> get another pair of eyeballs over it :) I look forward to any
>>> comments and if there are any red flags! If everything goes
>>> smoothly I will upload this, and do a larger area.
>>>
>>> Here is the link to the repo for anybody who wants to
>>> contribute:
>>> https://gitlab.com/dionmoult/osm-nsw-address-import/
>>> <https://gitlab.com/dionmoult/osm-nsw-address-import/>
>>> (because Github is proprietary).
>>>
>>> Dion Moult
>>>
>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>
>>>
>>> On June 27, 2018 8:04 AM, Dion Moult <dion at thinkmoult.com
>>> <mailto:dion at thinkmoult.com>> wrote:
>>>
>>> > Thanks for the response Andrew! Absolutely, sometimes
>>> government information is wrong, sometimes we represent it in
>>> a better way, etc: this method allows the mapper to make a
>>> judgement on a small case by case of exactly what to do.
>>> >
>>> > Probably something I didn't explicitly mention before, but
>>> of course as I add address information I plan to do it slowly
>>> and post it on talk-au for regular review and local QA.
>>> >
>>> > If there aren't any issues I will create an import plan on
>>> the wiki this weekend, along with some data to review.
>>> >
>>> > Dion Moult
>>> >
>>> > -------- Original Message --------
>>> >
>>> > On 24 Jun 2018, 13:54, Andrew Harvey <
>>> andrew.harvey4 at gmail.com <mailto:andrew.harvey4 at gmail.com>>
>>> wrote:
>>> >
>>> > > Overall I'm okay with the approach you've outlined.
>>> > >
>>> > > > and then upload the changeset tagged as
>>> `source:import=NSW LPI Web Services`, this will allow anybody
>>> who is checking the history to know how this data was gotten.
>>> > >
>>> > > +1
>>> > >
>>> > > > There will be no automatic data overwriting or data
>>> conflicts.
>>> > >
>>> > > That's good! I commented on the talk-au list, but to
>>> expand on a few points, OSM is all about mapping what's on
>>> the ground. There are many cases where the government
>>> supplied address points database conflicts with information
>>> on the ground. The on the ground scenario is what should
>>> prevail in OSM, in my opinion. If anyone wants a government
>>> supplied address points database, OpenAddresess does a great
>>> job at collating that.
>>> > >
>>> > > Attached or at https://snag.gy/3jBX5a.jpg you can see a
>>> place where I've mapped out an address different to the NSW
>>> LPI Basemap, and different to GNAF, instead reflecting what's
>>> on the ground.
>>> > >
>>> > > There's also differing views of how to map addresses in
>>> OSM, eg. on an entrance node, on a building, on a residential
>>> land parcel, as an unconnected address node. Additionally
>>> addresses can be duplicated, eg. shops or offices with the
>>> same address.
>>> > >
>>> > > My point is, I'm not against importing this data, but I
>>> think we should pick and choose what makes sense to bring in,
>>> and do it in a way that doesn't scare mappers away from
>>> deleting or changing that imported address data when they
>>> find it doesn't represent what's on the ground.
>>> > >
>>> > > On 23 June 2018 at 21:15, Dion Moult <dion at thinkmoult.com
>>> <mailto:dion at thinkmoult.com>> wrote:
>>> > >
>>> > > > Good morning all!
>>> > > >
>>> > > > The state of New South Wales (where Sydney is) in
>>> Australia currently has very minimal address information. I
>>> am looking to speed up the mapping of addresses. These can be
>>> simply nodes which have addr:street and addr:housenumber tags
>>> at a bare minimum, but of course could be better things like
>>> part of a building.
>>> > > >
>>> > > > Currently, apart from local knowledge, addresses can be
>>> mapped using the LPI (Land and Property Information) Base Map
>>> that we have explicit permission provided by the NSW
>>> government to use.
>>> > > >
>>> > > >
>>> https://wiki.openstreetmap.org/wiki/Contributors#Australia
>>> <https://wiki.openstreetmap.org/wiki/Contributors#Australia>
>>> > > >
>>> > > >
>>> https://wiki.openstreetmap.org/wiki/Attribution/New_South_Wales_Government_Data
>>> <https://wiki.openstreetmap.org/wiki/Attribution/New_South_Wales_Government_Data>
>>> > > >
>>> > > > The map provides raster tiles of property boundaries,
>>> along with the housenumber of the property. It is a little
>>> ambiguous, however, at intersections which street the address
>>> belongs to. A mapper is currently able to create a node,
>>> interpret the base map background, and then tag it as necessary.
>>> > > >
>>> > > > However, we have explicit permission to use all of the
>>> LPI web services, and the service API allows us to do two things:
>>> > > >
>>> > > > 1. Ask the question "At this coordinate, what is the
>>> address?"
>>> > > >
>>> > > > 2. Ask the question "What is the coordinate of this
>>> address?"
>>> > > >
>>> > > > Therefore, by roughly choosing coordinates (JOSM has
>>> the Edit->Copy Coordinates feature) that we know is within
>>> property boundaries (the boundary is shown in the LPI Base
>>> Map in JOSM that we always have turned on anyway), we can use
>>> a small script that will query the LPI Web Services API, ask
>>> those two questions, and then automatically place nodes at
>>> the accurate government centroid of the address. We can then
>>> review the results manually, and then upload the changeset
>>> tagged as `source:import=NSW LPI Web Services`, this will
>>> allow anybody who is checking the history to know how this
>>> data was gotten. This speeds up the adding and tagging of
>>> nodes as the mapper does not need to manually type in data
>>> (which is prone to spelling mistakes, and the housenumber in
>>> the raster base map is not a high resolution and can be
>>> misread), does not need to interpret ambiguous intersections
>>> where street is not so clear, and will improve the quality of
>>> the node placement as it will be at the actual centroid of
>>> the property, not randomly to the mapper's liking.
>>> > > >
>>> > > > As you can see I am not proposing a huge data import,
>>> so there is no actual data to review. Just a semi-automation
>>> of the manual address tagging that we do now anyway. However
>>> by some interpretation it could be called an "import" as it
>>> is semi-automated node placement based off a government API
>>> query. It is up to the individual mapper to manually choose
>>> coordinates that they want to tag addresses at - so if the
>>> mapper sees that addresses have already been tagged, they
>>> will simply not query the LPI services for the address at
>>> that point. There will be no automatic data overwriting or
>>> data conflicts.
>>> > > >
>>> > > > I have discussed this on the talk-au mailing list, and
>>> the responses seem positive. It's quite a conservative
>>> proposal, after all. You can see the thread here:
>>> > > >
>>> > > >
>>> https://lists.openstreetmap.org/pipermail/talk-au/2018-June/011937.html
>>> <https://lists.openstreetmap.org/pipermail/talk-au/2018-June/011937.html>
>>> > > >
>>> > > > The small script along with sample data is shown here:
>>> > > >
>>> > > >
>>> https://gist.github.com/Moult/5821c74fb792b7afa5d758aebea68e40 <https://gist.github.com/Moult/5821c74fb792b7afa5d758aebea68e40>
>>> > > >
>>> > > > I did a small test of 17 nodes in this changeset to
>>> show how it would work. I manually reviewed it and I know
>>> this area based off local knowledge.
>>> > > >
>>> > > > https://www.openstreetmap.org/changeset/59909707
>>> <https://www.openstreetmap.org/changeset/59909707>
>>> > > >
>>> > > > Looking forward to your comments!
>>> > > >
>>> > > > Dion Moult
>>> > > >
>>> > > > _______________________________________________
>>> > > >
>>> > > > Imports mailing list
>>> > > >
>>> > > > Imports at openstreetmap.org
>>> <mailto:Imports at openstreetmap.org>
>>> > > >
>>> > > > https://lists.openstreetmap.org/listinfo/imports
>>> <https://lists.openstreetmap.org/listinfo/imports>
>>
>
>
>
> _______________________________________________
> Imports mailing list
> Imports at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>
More information about the Imports
mailing list