<div dir="ltr">Sarah, how would I set the node cache file to the repserv.apply_diffs()? The idx param is passed to the apply_file() - for the initial PBF dump parsing, but I don't see any place to pass it for the subsequent diff processing. I assume there must be a way to run .apply_diff() that will download the minute diff file, update node cache file with the changed nodes, and afterwards call my way handler with the updated way geometries.<div><br></div><div>Also, I assume you meant dense_file_array, not dense_file_cache. So in my case I would use one of these idx values when calculating way centroid, and None otherwise:</div><div>sparse_mem_array</div><div>dense_mmap_array</div><div><div>sparse_file_array,my_cache_file<br></div><div></div><div>dense_file_array,my_cache_file</div></div><div><br></div><div>Thanks!</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 14, 2017 at 4:31 PM, Sarah Hoffmann <span dir="ltr"><<a href="mailto:lonvia@denofr.de" target="_blank">lonvia@denofr.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Mon, Aug 14, 2017 at 11:10:39AM -0400, Yuri Astrakhan wrote:<br>
> mmd, the centroids are calculated with this code, let me know if there is a<br>
> better way, I wasn't aware of any issues with the minute data updates.<br>
> wkb = wkbfab.create_linestring(obj)<br>
> point = loads(wkb, hex=True).representative_<wbr>point()<br>
> <a href="https://github.com/nyurik/osm2rdf/blob/master/osm2rdf.py#L250" rel="noreferrer" target="_blank">https://github.com/nyurik/<wbr>osm2rdf/blob/master/osm2rdf.<wbr>py#L250</a><br>
<br>
</span>It doesn't look like you have any location cache included when<br>
processing updates, so that's unlikely to work.<br>
<br>
Minutely updates don't have the full node location information.<br>
If a way gets updated, you only get the new list of node ids.<br>
If the nodes have not changed themselves, they are not available<br>
with the update.<br>
<br>
If you need location information, you need to keep a persistent<br>
node cache in a file (idx=dense_file_cache,file.<wbr>nodecache)<br>
and use that in your updates as well. It needs to be updated<br>
with the fresh node locations from the minutely change files<br>
and it is used to fill the coordinates for the ways.<br>
<br>
Once you have the node cache, you can get the geometries for<br>
updates ways. This is still only half the truth. If a node in<br>
a way is moved around, then this will naturally change the<br>
geometry of the way, but the minutely change file will have<br>
no indication that the way changed. Normally, these changes are<br>
relatively small and for some applications it is good enough<br>
to ignore them (Nominatim, the search engine, does so, for example).<br>
If you need to catch that case, then you also need to keep a<br>
persistent reverse index of which node is part of which way<br>
and for each changed node, update the ways it belongs to.<br>
There is currently no support for this in libosmium/pyosmium.<br>
So you would need to implement this yourself somehow.<br>
<br>
Kind regards<br>
<span class="HOEnZb"><font color="#888888"><br>
Sarah<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
><br>
> Your query is correct, and you are right that (in theory) there shouldn't<br>
> be any ways without the center point. But there has been a number of ways<br>
> with only 1 point, causing a parsing error "need at least two points for<br>
> linestring". I will need to add some special handling for that<br>
> (suggestions?).<br>
><br>
> You can see the error by adding this line:<br>
> OPTIONAL { ?osmId osmm:loc:error ?err . }<br>
> The whole query -- <a href="http://tinyurl.com/ydf4qd62" rel="noreferrer" target="_blank">http://tinyurl.com/ydf4qd62</a> (you can create short urls<br>
> with a button on the left side)<br>
><br>
> On Mon, Aug 14, 2017 at 5:18 AM, mmd <<a href="mailto:mmd.osm@gmail.com">mmd.osm@gmail.com</a>> wrote:<br>
><br>
> > Hi,<br>
> ><br>
> > Am 13.08.2017 um 19:49 schrieb Yuri Astrakhan:<br>
> ><br>
> > > * all ways now store "osmm:loc" with centroid coordinates, making it<br>
> > > possible to crudely filter ways by location<br>
> ><br>
> > out of curiosity, can you say a few words on how your overall approach<br>
> > to calculate centroids for ways? As we all know it's an endless pain to<br>
> > get that information out of minutely diffs :)<br>
> ><br>
> > I have to say that I'm pretty much unfamiliar with SPARQL and just tried<br>
> > the following query. My expectation was that I won't get any results,<br>
> > making me wonder if my query has some issue?<br>
> ><br>
> > SELECT * WHERE {<br>
> > ?osmId osmm:type 'w' .<br>
> > FILTER NOT EXISTS { ?osmId osmm:loc ?osmLoc }.<br>
> > } LIMIT 100<br>
> ><br>
> ><br>
> > BTW: A quick search on Github yielded the following:<br>
> > <a href="https://github.com/nyurik/osm2rdf" rel="noreferrer" target="_blank">https://github.com/nyurik/<wbr>osm2rdf</a>. Would that be the right place to look<br>
> > for more details?<br>
> ><br>
> > Best,<br>
> > mmd<br>
> ><br>
> ><br>
> > --<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > talk mailing list<br>
> > <a href="mailto:talk@openstreetmap.org">talk@openstreetmap.org</a><br>
> > <a href="https://lists.openstreetmap.org/listinfo/talk" rel="noreferrer" target="_blank">https://lists.openstreetmap.<wbr>org/listinfo/talk</a><br>
> ><br>
<br>
> ______________________________<wbr>_________________<br>
> talk mailing list<br>
> <a href="mailto:talk@openstreetmap.org">talk@openstreetmap.org</a><br>
> <a href="https://lists.openstreetmap.org/listinfo/talk" rel="noreferrer" target="_blank">https://lists.openstreetmap.<wbr>org/listinfo/talk</a><br>
<br>
</div></div></blockquote></div><br></div>