[Imports] Proposal:Uganda Import

Jo winfixit at gmail.com
Fri Jan 31 20:54:27 UTC 2014


Hi,

I put the resulting osm file here:

https://dl.dropboxusercontent.com/u/42418402/Uganda%20Complete.zip

I've vetted it manually and applied Francisco's algorithm on it in order to
eliminate 900 unnecessary nodes.

So now it's ready to add the data to OSM. I guess the easiest way to
accomplish that is to work from a region that is located in the center of
the country. Then work from the inside outwards, leaving the hardest part
(integration with the existing national boundaries in osm) for the end.

This morning we had a hangout and then I demonstrated how working from the
outside inwards could be done. The problem is that it's all connected
(there are other boundaries like national parks and in some places roads
and rivers are used as part of the boundaries), so it would take a long
time before anything could be uploaded. Say it takes a few weeks to prepare
it all, then the risk is that there will be many conflicts to resolve,
which is rather daunting.

I'm not sure how to go about resolving the differences with boundaries that
apparently came from a previous import of a few years ago. We'd need a
third source to compare, or maybe more recent data from the neighbouring
countr(y|ies).

Polyglot


2014-01-31 Ketty Adoch <kettya298 at gmail.com>:

> Hi all,
>
> Thanks for your excellent work on all this- we now have an osm file with
> 123 relations as reflected  in the merged shapefile. The one set back we
> see at this stage is dealing with the national boundaries(osm data and
> generated osm file from Uganda shapefile): they are not in sync(+/- 200
> meters apart), and even if they were we wouldn't want duplicate ways in the
> database.
>
> Jo and I had a discussion on the best approach to do this, here are our
> suggestions:
>
> -Go through the entire length of the national boundary and compare the two
> layers, merge and join where necessary.
>
> -Or, start from the centre of the layer and work outwards, since there are
> no boundaries to merge.
>
> -Then we'd have to be careful with the upload ensuring that only the
> verified boundaries get uploaded(by making a selection of these).
>
> What are your thoughts on this approach?
>
> Francisco, is it possible to automate the boundary syncing for the Uganda
> case?
>
> regards,
>
> Ketty.
>
> Ketty Adoch
> Mobile:  +256-712-200-738
> Tweet @kadoch | Skype adockatie
>
>
>
> On Thu, Jan 30, 2014 at 5:13 PM, Ketty Adoch <kettya298 at gmail.com> wrote:
>
>> Ah, Francisco, I got it.
>>
>> Sorry for all the questions, and thanks for your patience. I can gladly
>> spam the imports list now ;-)
>>
>> Ketty
>>
>>
>>
>> Ketty Adoch
>> Mobile:  +256-712-200-738
>> Tweet @kadoch | Skype adockatie
>>
>>
>>
>> On Thu, Jan 30, 2014 at 5:09 PM, Ketty Adoch <kettya298 at gmail.com> wrote:
>>
>>> Yeah you are right, was installing from source, python-gdal works
>>> perfectly.
>>>
>>> So everything goes well though I am wondering where the generated osm
>>> file is placed after executing the script?
>>>
>>> Thanks,
>>>
>>> Ketty.
>>>
>>>
>>>
>>> Ketty Adoch
>>>  Mobile:  +256-712-200-738
>>> Tweet @kadoch | Skype adockatie
>>>
>>>
>>>
>>> On Thu, Jan 30, 2014 at 1:12 PM, <f.dos.santos at free.fr> wrote:
>>>
>>>> Hi,
>>>>
>>>> You'll need the python bindings for gdal, it comes from gdal but for my
>>>> linux distro (Fedora) it's packaged separatly, in my case the package is
>>>> named gdal-python.
>>>>
>>>> And you don't need postgresql, postgis, psycopg2 anymore, now I save to
>>>> an .osm file instead of the database (but I haven't updated the README
>>>> file). Just give the filename.shp and it will create a filename.osm :-)
>>>>
>>>>
>>>> Francisco
>>>>
>>>> ----- Mail original -----
>>>> De: "Ketty Adoch" <kettya298 at gmail.com>
>>>> À: "f dos santos" <f.dos.santos at free.fr>, "Aggrey Muhebwa" <
>>>> muhebwa.grey at gmail.com>
>>>> Envoyé: Jeudi 30 Janvier 2014 09:45:46
>>>> Objet: Re: [Imports] Proposal:Uganda Import
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Hi Francisco,
>>>>
>>>> Thank you for the excellent work you've done with this Uganda osm file.
>>>> I did a git clone, installed
>>>>
>>>> - Python 2.7.3
>>>> - gdal 1.7.3
>>>> - postgresql 9.1.3
>>>> - postgis 1.5.3
>>>> - psycopg2 2.4.5
>>>>
>>>>
>>>> cd into uganda_osm and run the script as suggested in the readme. I get
>>>> this error:
>>>>
>>>> "Traceback (most recent call last):
>>>> File "uganda_build.py", line 28, in <module>
>>>> from osgeo import gdal, ogr, osr"
>>>>
>>>> Secondly is there anything i need to do to create an osmosis schema.
>>>>
>>>> #Will revert to the imports mailing list after this little inquiry ;-)
>>>>
>>>>
>>>> Ketty.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Ketty Adoch
>>>> Mobile: +256-712-200-738
>>>>
>>>> Tweet @kadoch | Skype adockatie
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Jan 28, 2014 at 3:57 AM, < f.dos.santos at free.fr > wrote:
>>>>
>>>>
>>>> Hello,
>>>>
>>>> I promise, this is my last attempt to convert this shapefile ;-)
>>>>
>>>> This time this is not 1 but 2 OSM file, I've put in [1] a zip with both
>>>> source and result :
>>>> - Uganda_districts2010.shp this is the (112) district only file
>>>> - districtReSimRenamed.shp the merge 11 region + 112 district
>>>>
>>>> The code in [2] have been adapted to work with those type of files, it
>>>> does a better job dealing with the "very close but not the same point"
>>>> problem and doesn't require a database anymore.
>>>> This works much better now, and it can build automagically the admin
>>>> hierarchy :-)
>>>>
>>>> So you've got 2 result OSM file :
>>>> - districtReSimRenamed.osm with the 123 shapes converted into admin
>>>> boundary relations
>>>> - Uganda_districts2010.osm with the 112 district shapes converted into
>>>> 123 admin boundary relations
>>>>
>>>> There is still errors in those files, I'll list just a few.
>>>>
>>>> Look for example in the Uganda_districts2010.osm, in region Western
>>>> (relation -46) you'll see border Kyenjojo and Kibaale have a lot of
>>>> parallel but not the same way :
>>>> - way -341 and way -425
>>>> - way -342 and way -424
>>>>
>>>> For Kabale (relation -70) and Kanungu (relation -60) :
>>>> - way -377 and way -239
>>>>
>>>> For Kanungu (relation -60) and Rukungiri (relation -40)
>>>> - way -376 and way -369
>>>> Please note that the split for way -363 is not an error, it was
>>>> splitted before reaching the 2000 nodes limits.
>>>>
>>>> Kaberamaido (relation -90)
>>>> - way -126 very small loop that the simplification process squashed
>>>> into a single line
>>>>
>>>> In the districtReSimRenamed.osm you'll see the same errors and a new
>>>> one : Teso (relation -6) and district doesn't share the same ways.
>>>> - in way -101 versus way -140 and -141 for district, there is an offset
>>>> about 60m
>>>> - way -98 versus -150 for district, there is an offset about 100m
>>>>
>>>> And finally, the admin level number are obviously wrong as I've chosen
>>>> them only as example for the purpose of the conversion process.
>>>>
>>>> Francisco
>>>>
>>>> [1] http://dl.free.fr/mC6bXQBK2
>>>> [2] https://github.com/FranciscoDS/uganda_osm.git
>>>>
>>>>
>>>> ----- Mail original -----
>>>> De: "f dos santos" < f.dos.santos at free.fr >
>>>> À: "Imports OpenStreetMap.org" < imports at openstreetmap.org >
>>>> Envoyé: Samedi 25 Janvier 2014 16:36:50
>>>>
>>>> Objet: Re: [Imports] Proposal:Uganda Import
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I have done some work to prepare an import (not discussed yet on this
>>>> list as it's still a work on progress) of portuguese administrative area.
>>>> I tested my code on the Uganda shapefile and, after correcting some
>>>> issues, I've a result (far from perfect as the source is far from perfect)
>>>> that can be useful.
>>>>
>>>> The OSM file is here [1] and the (experimental) python code is [2].
>>>>
>>>> The code have been adapted to the uganda shapefile, and does the
>>>> following :
>>>> - build boundary relation for each admin area
>>>> - build hierarchy of admin area from the lower admin area (district)
>>>> build the upper (subregion) : DEACTIVATED as it's not working well for this
>>>> shapefile
>>>> - capitalize names and convert from ISO to UTF8
>>>> - construct unique way, simplify it and deal with the 2000 nodes limit
>>>> - detect and try to resolve self intersecting way
>>>> - detect enclave and exclave and give them the proper outer/inner role
>>>> (for Uganda there is no inner ring)
>>>> - save the result on a local postgres+postgis database
>>>>
>>>> The OSM file still contains "issues" so it can not be used as is, but
>>>> can probably be edited and fixed manually :
>>>> - merging the suboptimal ways (two identical ways but not sharing
>>>> exactly the same nodes coordinates).
>>>> - adjusting the admin_level
>>>>
>>>> There is still one big issue in this OSM file : I removed 1 district
>>>> because of a loop in the shapefile (and my code goes into an infinite loop).
>>>>
>>>> I hope it can help.
>>>>
>>>> Francisco.
>>>>
>>>> [1] dl.free.fr/mpSmh2T9m
>>>> [2] https://github.com/FranciscoDS/uganda_osm.git
>>>>
>>>>
>>>> 2014/01/21 "Satoshi IIDA" < nyampire at gmail.com >
>>>>
>>>> >Hi,
>>>> >
>>>> >I made a .osm file but they are NOT relationalized. :)
>>>> >So please do not upload them directory.
>>>> > https://www.dropbox.com/sh/89qkkql46k61zj0/W5xTa9zW4h
>>>> >
>>>> >I think it is better to wait Jo's work.
>>>> >
>>>> >> source file
>>>> >
>>>> >seems contain some errors.(e.g. self crossing ways, suspicious small
>>>> enclave, or so)
>>>> >
>>>> >might need some validation step, in case of.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 2014/1/22 Jo < winfixit at gmail.com >
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I'm looking into converting your shapefiles into proper OSM relations
>>>> automatically. It will take a while (a few days) though to finish the
>>>> script.
>>>>
>>>> I sent you a file yesterday, but it's better to wait a bit. That was
>>>> the result of some manual practice to find out how to automate it.
>>>>
>>>> Polyglot
>>>>
>>>>
>>>>
>>>>
>>>> 2014/1/21 Aggrey Muhebwa < muhebwa.grey at gmail.com >
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Hello folks,
>>>>
>>>>
>>>> Following Ketty's proposal for the Uganda imports , i decided to start
>>>> by running the shape file for one district (specifically dokolo district),
>>>> but for some reason, the generated .osm file has <way>....<.way> instead of
>>>> <relation>...</relation>. Somewhere, she indicates that of 112 districts,
>>>> only a couple of them are being processed!! Could it be an issue of wrong
>>>> data in the shape file that is being fed to the ogr2osm.py script??
>>>>
>>>>
>>>> More to that, along the iway in the discussion, someone(my apologies
>>>> for not mentioning the name) states that there should be a work-around to
>>>> that, but that's unclear, given the fact that the ogr2osm.py algorithm
>>>> follows specific steps (and it doesn't seem to be handling checks for worst
>>>> case scenarios), then it will always give you wrong output if you feed it
>>>> wrong imput.. (garbage in, garbage out..).
>>>>
>>>>
>>>> My proposal is to make sure that the shape file at the input is
>>>> correct. is there any way of validating the shape file before running it
>>>> through the script ?.
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>> Yours Faithfully
>>>> Aggrey Muhebwa
>>>> (+256)774561760 /701909434
>>>> Twitter:@m_aggrey
>>>>
>>>> Skype:Aggrey Robotboy
>>>> IRC: AggreyMuhebwa-Robotboy
>>>> "Creativity is Intelligence having fun"
>>>> _______________________________________________
>>>> Imports mailing list
>>>> Imports at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/imports
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Imports mailing list
>>>> Imports at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/imports
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Satoshi IIDA
>>>> mail: nyampire at gmail.com
>>>> twitter: @nyampire
>>>> _______________________________________________
>>>> Imports mailing list
>>>> Imports at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/imports
>>>>
>>>> _______________________________________________
>>>> Imports mailing list
>>>> Imports at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/imports
>>>>
>>>> _______________________________________________
>>>> Imports mailing list
>>>> Imports at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/imports
>>>>
>>>>
>>>
>>
>
> _______________________________________________
> Imports mailing list
> Imports at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20140131/764a77ca/attachment-0001.html>


More information about the Imports mailing list