<div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div dir="ltr" class="gmail_signature"><div dir="ltr">jared,</div><div dir="ltr"><br></div><div>I see how the source could be useful with that specific Overpass query but also have a better option that will let you get more information from overpass. It is very simple.</div><div><br></div><div>Instead of using the command 'out body', use 'out meta'. The meta includes all available metadata. In it I can see the changeset, user, and version of every node. That should help you narrow it down.</div><div><br></div><div>Example query: <a href="https://overpass-turbo.eu/s/1my4">https://overpass-turbo.eu/s/1my4</a></div><div><br></div><div>Example output:</div><div><br></div><div><div> "type": "node",</div><div> "id": 8825645506,</div><div> "lat": 44.0520033,</div><div> "lon": -73.3760089,</div><div> "timestamp": "2021-06-11T16:30:27Z",</div><div> "version": 1,</div><div> "changeset": 106224744,</div><div> "user": "jared",</div><div> "uid": 3887,</div><div> "tags": {</div><div> "addr:city": "Addison",</div><div> "addr:housenumber": "1245",</div><div> "addr:postcode": "05491",</div><div> "addr:state": "VT",</div><div> "addr:street": "Jersey Street South",</div><div> "source": "esri/USA_NAD_Addresses"</div><div> }</div></div><div dir="ltr"><br></div><div dir="ltr">- Elliott</div></div></div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 7, 2022 at 10:30 AM Jared <<a href="mailto:osm@wuntu.org">osm@wuntu.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Elliott or Greg, <div><br></div><div>Can you walk me through a real example so I can understand how you would identify existing addresses?</div><div><br></div><div>Let's take Addison, Vermont for example.</div><div><br></div><div>The VCGI e911 dataset has 987 address points in Addison. Here's the data file: </div><div><a href="https://github.com/JaredOSM/vermont-address-import/blob/main/town_e911_address_points/e911_address_points_addison.geojson" target="_blank">https://github.com/JaredOSM/vermont-address-import/blob/main/town_e911_address_points/e911_address_points_addison.geojson</a></div><div><br></div><div>When I run an overpass query for all elements in Addison that have a housenumber or street: <a href="https://overpass-turbo.eu/s/1mxX" target="_blank">https://overpass-turbo.eu/s/1mxX</a></div><div>I find that there are already a total of 142 nodes and ways with address information OSM. </div><div><br></div><div>By looking at the overpass results, I can immediately see that 55 of the existing OSM elements have a "ref:vcgi:esiteid" Key/Value pair. Without any further queries, I have a high level of confidence that I can remove all 55 address points from my import file, as they are not even worth considering for an automated import. This seems like a safe and efficient way of eliminating the chance of importing duplicate data. Obviously the other data points need to be evaluated, but why not remove the 55 for which I have high confidence?</div><div><br></div><div>Thanks for helping walk me through how you would approach it, or explain why my technique could be flawed.</div><div>Jared</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 7, 2022 at 9:50 AM Elliott Plack <<a href="mailto:elliott.plack@gmail.com" target="_blank">elliott.plack@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Jared,<div><br></div><div>This looks great! I want to thank you for the due diligence. The process looks sound.</div><div><br></div><div>I do agree about the source tags on the nodes, they may not be as reliable. In my experience I check the editor/history of a node for authority and if I saw it was made via an import account, I might hold it to a different standard--not a bad thing. If you are concerned about downstream querying of previously imported addresses, you can query out things from the import using the changeset (keep a record), user, or version with overpass. I'd recommend looking at that option.</div><div><br></div><div>Otherwise I applaud the effort.</div><div><br></div><div>Thanks,</div><div><br clear="all"><div><div dir="ltr"><div dir="ltr">Elliott Plack</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 7, 2022 at 8:33 AM Greg Troxel <<a href="mailto:gdt@lexort.com" target="_blank">gdt@lexort.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Jared <<a href="mailto:osm@wuntu.org" target="_blank">osm@wuntu.org</a>> writes:<br>
<br>
> On Thu, Oct 6, 2022 at 8:13 AM Greg Troxel <<a href="mailto:gdt@lexort.com" target="_blank">gdt@lexort.com</a>> wrote:<br>
><br>
>> You are going to have to deal witha matching addresses between import<br>
>> source and OSM programmatically like in #1 above, once you move beyond<br>
>> non-addressed towns. Once you do that, the ref won't help, as it won't<br>
>> be 100% reliable. Therefore it's noise.<br>
><br>
> I was thinking of using the foreign key for a different use case. I agree<br>
> that relying on this key for *overwriting* OSM data does not seem safe.<br>
> The scenario I'm thinking about is for NEW addresses that are added to the<br>
> VCGI dataset. To determine if a NEW VCGI e911 address exists in OSM, the<br>
> "ref:vcgi:esiteid" tag would seem to be very helpful. If an address in OSM<br>
> already has that unique esiteid key, then we can be confident that it<br>
> should be skipped. If the esiteid does not exist in OSM, then other<br>
> signals should be evaluated (housenumber, streetname, lat/long, etc., but<br>
> those can be less precise due to misspellings or slightly different<br>
> coordinates.<br>
<br>
I see where you're going but I think you need to get the fuzzy match<br>
right anyway and it's not going to help that much to have a key.<br>
<br>
> I'd like to hear the negative impact a foreign key causes. There are other<br>
> similar foreign keys (eg. wikidata, wikipedia) and I've never found them to<br>
> be detrimental to my work, but don't want to cause issues for others. The<br>
> 55,000 VT addresses that have been added using the Esri layer in the RapiD<br>
> editor include this "ref:vcgi:esiteid" key, and I've found it to be useful.<br>
<br>
A fair question, and it may be that the RapiD stuff is out of line.<br>
<br>
I don't think the foreign keys really hurt. I just think that the<br>
history is that they are less useful than everybody thinks they are going<br>
to be.<br>
<br>
>> Wow. Are you saying that apartment buildings have coordinates of entry<br>
>> doors within the building, or that they are artificially skewed to make<br>
>> rendering non-overlapping, or ? Surely Vermont has at least some<br>
>> multi-floor apartment buildings that have the same floor design and thus<br>
>> multiple units that actually do have the same horizontal coordinates.<br>
><br>
> I've asked my contact at VCGI for clarification on how multi-tenant<br>
> buildings are addressed. From what I've seen, some multi-tenat buildings<br>
> just have one e911 address associated with them. I have seen other<br>
> buildings that have multiple addresses, but I've never seen them overlap.<br>
> I'll keep a close eye out for this and will see what VCGI has to say. I do<br>
> have the VT data in a postgis database, but don't have experience using the<br>
> GIS functions, so I'll try it out.<br>
<br>
Sounds good. There are hard questions about datasets and as you can see<br>
my bias is to dig in and address them.<br>
_______________________________________________<br>
Imports-us mailing list<br>
<a href="mailto:Imports-us@openstreetmap.org" target="_blank">Imports-us@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/imports-us" rel="noreferrer" target="_blank">https://lists.openstreetmap.org/listinfo/imports-us</a><br>
</blockquote></div>
</blockquote></div>
</blockquote></div>