[OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

Andy Townsend ajt1047 at gmail.com
Fri Sep 29 12:40:43 UTC 2017


On 28/09/2017 07:15, Stefano wrote:
>
>
> We used this library to process the dump and then we add the results 
> in pgsql
>
> https://www.entropywins.wtf/blog/2015/11/08/wikidata-wikibase-json-dump-reader/
>
> https://github.com/osmItalia/wikidata-geo-match
>
> What would the requirement of a wikidata2pgsql be?

Thanks for that.

To answer the question, It'd create a database in a format that's 
designed to be queried that contains "just enough" information to 
support whatever job it's needed for (and it'd be great if it also 
supported dynamic column creation using a mechanism similar to 
osm2pgsql's ".style" file).

Of the other "missing bits", I used "osmosis" as an example of "cutting 
a database extract down to size" (other options are available).  In the 
OSM world that initial slice is often geographical, but osmosis can also 
deal with data without explicit co-ordinates (ways and relations) based 
on the geographical location of constituent nodes.  The same would be 
true (for me) of wikidata - I'd be only interested in actual physical 
locations and the things that they link to (which may not have physical 
locations and may just be concepts).

Finally "switch2osm" is a regularly-updated set of instructions that you 
can follow from start to finish without needed external knowledge about 
how to solve a problem.  For example, 
https://switch2osm.org/manually-building-a-tile-server-16-04-2-lts/ has 
undergone numerous updates in the last year to deal with stylesheet 
changes, which depend on a bleeding edge version of carto, which depends 
on <BAD_WORD> node.js.  At each stage in the process the idea was that 
you'd always be able to get a working result, even if at one point that 
meant the instructions explained how get a version of the stylesheet 
from a few months ago because a newer version wouldn't work in 
combination with everything else there.

"wikidata-wikibase-json-dump-reader" looks interesting - it looks (to 
continue the analogy) to be somewhat equivalent to the Crosby PBF library.

"wikidata-geo-match" also looks interesting because 
https://github.com/osmItalia/wikidata-geo-match/blob/master/scripts/2_process.sh 
and the readme explain how to do the initial geographical selection.  
It's not quite all there though 
(https://github.com/osmItalia/wikidata-geo-match/search?utf8=%E2%9C%93&q=0_create_wikidata_table.sql&type= 
suggests either the README is out of date or some bits are missing).  
It'd certainly be a useful start for someone who cared about wikidata to 
develop something mirroring the equivalent tools that OSM already has, 
and I'm sure it does exactly what you need it to do, but it's not a 
generic "let's create a database and allow you to do something with it".

Best Regards,

Andy




More information about the talk mailing list