[talk-au] Mapping houses and addresses in Sydney

Dion Moult dion at thinkmoult.com
Mon Jun 18 09:21:23 UTC 2018


Thanks Andrew for your reply!

1. Thanks for the link to the import guidelines. My responses to the import guidelines below:

 1. I am aware that big automatic updates can cause problems. I will only import addr:housenumber and addr:street and a single node.
 2. Yep, sending to talk-au mailing list.
 3. I think we have the appropriate license.
 >=4. Looking for more feedback on talk-au first :)

2. Yes, you are absolutely right that this is not a huge automatic import - it relies on a human choosing what addresses to add and a human submitting it as a change. All it does it automate the address lookup and make sure that the node is neatly positioned at the correct location.
3. It looks like you're grabbing their entire dataset. That would be the alternative approach, doing a data dump, then importing that dump. This can import a lot more addresses, but is also much more complex. Is it worth pursuing? What do you reckon?
4. It seems odd that they would provide an API but would prevent anything from using it.
5. Looks like they are doing the big data import. See 3.

Dion Moult

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On June 17, 2018 9:57 PM, Andrew Harvey <andrew.harvey4 at gmail.com> wrote:

> I few thoughts:
>
> 1. If you're planning an Import, and you haven't already read up on https://wiki.openstreetmap.org/wiki/Import/Guidelines
>
> 2. I like the idea of doing this semi-manually in JOSM. I'm against a completely automated import because we have addresses already in OSM from surveys which should be retained (and avoid having duplicates dumped on top).
>
> 3. I know you've got your scripts working already, but you still might be interested in how we're trying to pull out addresses via the LPI Web Services as CC BY 3.0 AU data for OpenAddresses.io at https://github.com/openaddresses/openaddresses/pull/3977
>
> 4. We are still bound by the Terms and Conditions of using the web services at http://spatialservices.finance.nsw.gov.au/mapping_and_imagery/lpi_web_services/terms_and_conditions specifically "2.1.6. use any robot, spider, site search/retrieval application, or other device to retrieve or index any portion of the SWS". I've asked LPI about this before and was told it was okay as long as usage didn't affect other users, they were thankful that I asked first though. The way you're using the API I think is okay, but best to play it safe and be friendly with LPI on what we're doing in light of this clause.
>
> 5. The NZ community have been importing their national LINZ data, might be worth reading up on https://lists.openstreetmap.org/pipermail/imports/2017-June/005014.html to see if there's anything we could learn from their work.
>
> On 17 June 2018 at 19:00, Dion Moult <dion at thinkmoult.com> wrote:
>
>> I had another look at the LPI resource and would like to propose a way of importing address data from LPI, because clearly sitting there for months on end armchair mapping the LPI Base Map as a raster underlay in JOSM is incredibly inefficient. Also, given that I am proposing an import, I would really appreciate community feedback to make sure I don't do anything wrong. First, I've looked at these two pages on usage of the LPI base map:
>>
>> https://wiki.openstreetmap.org/wiki/Contributors#Australia
>> https://wiki.openstreetmap.org/wiki/Attribution/New_South_Wales_Government_Data
>>
>> It seems to me that we can use any of the data from http://spatialservices.finance.nsw.gov.au/mapping_and_imagery/lpi_web_services/access_lpi_web_services to put data into OSM as long as we add it to that contributors page and note the LPI as a source. Specifically, we want to import a node that contains addr:housenumber and addr:street for every single property in NSW.
>>
>> Unfortunately it seems as though their API doesn't have any function to give a DB dump of addresses inside a bounding box. However it does have two functions, one which allows you to query a coordinate and it'll tell you what is the closest address at that point at a screen resolution (slightly archaic, yes), and another function which allows you to submit an address and it'll tell you what the government thinks is the centroid of the property with that address. So if you whack a bunch of points in JOSM that are within the bounds of a property based off the LPI NSW Base Map as a raster layer in JOSM, you can do Edit->Copy Coordinates in JOSM, dump it in a file, and use a script I wrote which will query the LPI web services to find the address, then query it again to find the proper centroid lat/long, then create a CSV of points and addresses which you can import into JOSM. Here's the script with sample input and sample output:
>>
>> https://gist.github.com/Moult/5821c74fb792b7afa5d758aebea68e40
>>
>> ... this is the changeset I produced with this small test sample:
>>
>> https://www.openstreetmap.org/changeset/59909707
>>
>> With this method I reckon you can quite quickly and accurately import addresses for a suburb. Note that this only imports nodes. It is up to somebody else to draw buildings and so on. Note that this uses two web services, the NSW Address Location Service and the NSW Property Service. The former is already credited in the contributors page, but the latter is not yet - it will have to be added, I suspect.
>>
>> Looking forward to any feedback!
>>
>> Dion Moult
>>
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On June 17, 2018 3:14 PM, Daniel O'Connor <daniel.oconnor at gmail.com> wrote:
>>
>>> For knowing "whats mapped" at a high level, I like http://qa.poole.ch - I believe it has filters for unmapped addresses.
>>>
>>> On the ground stuff - streetcomplete does really well at prompting for things without address, highly recommend.
>>>
>>> Setting up a task manager instance; I reckon it could make a lot of sense for us as a community. We seem to be a small group who punch above our weight in terms of how spread out we are but have much we have mapped.
>>> Alternatively, maproulette tasks.
>>>
>>> It seems to work well for US imports of address/buildings/etc; and its reasonable to expect that:
>>> - One day; the gnaf will be suitable for import or
>>> - LPI Basemap NSW addresses or otger data.gov.au imports for an entire town/area/etc make sense.
>>> - There will be communities within the AU mapping crowd with focuses (camping; cycling; tourism; public transport spring to mind)
>>>
>>> Looking at https://github.com/hotosm/tasking-manager/blob/develop/README.md it seems like python/postgres are the main server hosting requirements.
>>>
>>> On Sun, 17 Jun. 2018, 11:17 am Dion Moult, <dion at thinkmoult.com> wrote:
>>>
>>>> Wow, seems a little overkill to put the task there where it seems the objective is for humanitarian needs :) Australian house addresses are hardly humanitarian in nature :)
>>>>
>>>> Dion Moult
>>>>
>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>> On June 17, 2018 9:30 AM, Phil Wyatt <phil at wyatt-family.com> wrote:
>>>>
>>>>> You could put a project up on the HOT Task manager as a low priority and then send folks the link
>>>>>
>>>>> https://tasks.hotosm.org/contribute?difficulty=ALL
>>>>>
>>>>> Cheers - Phil,
>>>>> On the road with his iPad
>>>>>
>>>>> On 16 Jun 2018, at 10:29 pm, Dion Moult <dion at thinkmoult.com> wrote:
>>>>>
>>>>>> As a continuation of this thread just wondering if there was a way to divide up Sydney into a grid and automatically check the progress of whether that zone has been mapped or not. Is there a standard approach for this?
>>>>>>
>>>>>> Dion Moult
>>>>>>
>>>>>> -------- Original Message --------
>>>>>> On 6 Jun 2018, 09:39, Warin < 61sundowner at gmail.com> wrote:
>>>>>>
>>>>>>> On 06/06/18 08:51, Dion Moult wrote:
>>>>>>>> Thanks Andrew et all,
>>>>>>>>
>>>>>>>> Yes, I have used the ESRI imagery layer. Thanks for the heads up of the LPI base map. Before I go too far, can one of you please look at the work I've done so far if I'm doing things correctly? Here is a small sample in the suburb of Epping that I have traced houses and added addresses:
>>>>>>>>
>>>>>>>> https://www.openstreetmap.org/#map=18/-33.78071/151.06628
>>>>>>>>
>>>>>>>> In that scenario I did not trace off the LPI imagery, but instead traced off ESRI. I noticed if I enable the LPI imagery, the houses don't line up. Should this be fixed? If so, what's the best way?
>>>>>>>
>>>>>>> Any imagery can have a 'offset' from where it should be. And that offset will vary as you go up and down hills.
>>>>>>>
>>>>>>> The LPI Imagery has much better resolution and less parallax error than the other sources .. I prefer it to any other images for its accuracy.
>>>>>>>
>>>>>>> That said it is usefull to check with other images for more up to date things. I found in Coffs Harbour that the DigitalGlobe stuff is more current.
>>>>>>>
>>>>>>> I'll leave others to comment on that address stuff.. no expert on that (yet). :)
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Talk-au mailing list
>>>>>>> Talk-au at openstreetmap.org
>>>>>>> https://lists.openstreetmap.org/listinfo/talk-au
>>>>>
>>>>>> _______________________________________________
>>>>>> Talk-au mailing list
>>>>>> Talk-au at openstreetmap.org
>>>>>> https://lists.openstreetmap.org/listinfo/talk-au
>>>>
>>>> _______________________________________________
>>>> Talk-au mailing list
>>>> Talk-au at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/talk-au
>>
>> _______________________________________________
>> Talk-au mailing list
>> Talk-au at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-au/attachments/20180618/b9b5756d/attachment-0001.html>


More information about the Talk-au mailing list