[Imports] Update: Somalia populated places imported; working on roads

[Schuyler Erle]

TL;DR: I imported almost 9,500 populated place names into OSM from a 2010 UN dataset, after removing duplicates with existing OSM data and with the UN dataset itself. I would like to import the admin1 and admin-2 boundaries from the UN as well, and I would appreciate suggestions on how to proceed. HOT has also been donated a decent looking highway dataset for Somalia, and I have a plan for importing that, following a brainstorming session on #osm. Sorry for the crosspost.

As I mentioned in my previous email, HOT received permission from the UN Office for the Coordination of Humanitarian Affairs to import data from its "Common Operational Dataset" for Somalia (including Puntland and Somaliland), following the announcement of the famine crisis in the Horn of Africa.

  http://cod.humanitarianresponse.info/country-region/somalia

As of yesterday, there were 227 named places in Somalia in OpenStreetMap. There are 9,638 named places in the OCHA dataset. I flagged all the places across the two datasets within 15km of each other with a trigram similarity on any name greater than 1/3, or within 1.5km with a normalized Levenshtein edit distance less than 1/3, and then manually reviewed the result. I did the same within the OCHA dataset to remove obvious duplicate entries. I imported the remaining 9,489 places into OSM this evening, as the following change set:

  http://www.openstreetmap.org/browse/changeset/9062081

Please see my previous email to the list for how the data was structured:

  http://lists.openstreetmap.org/pipermail/imports/2011-August/001047.html

I would like to import the region and district boundaries from the OCHA COD next. Does anyone have any thoughts on how to go about this? If I just import the polygons as is, we'll wind up with a lot of redundant nodes and ways, I'm guessing.

Next, HOT has been been donated a decent looking highway dataset by MadMappers as described on the data sources page on the wiki.

I have reviewed this dataset on the HOT list:

  http://lists.openstreetmap.org/pipermail/hot/2011-August/000897.html
  http://iconocla.st/hot/so/madmappers1.png
  http://iconocla.st/hot/so/madmappers2.png

There are 753 roads currently in the OSM database in Somalia. There are 37,427 roads in the MadMappers dataset. The good folks on the #osm channel on IRC were kind enough to brainstorm with me for a bit on how to proceed. My current plan is to split the MapMappers road dataset into two pieces: One which contains all the roads that are no closer than 500m to an existing road in OSM. These I will import as-is, retaining information on provenance and road surface wherever possible.

All of the MadMappers road features within 500m of an existing road in OSM will be split up geographically, and the chunks will be converted into OSM files and posted on the wiki. I will put out a call for volunteers on the HOT list to "check" out these files, and integrate them manually using JOSM, taking care to attach the nodes at the edges. I would like to have all of this work completed by Monday, provided that I can find the volunteers. I will document everything on the wiki.

Does anyone have any comments? Thanks in advance.

SDE