[OSM-talk] HowTo : Import from .shp (ESRI) (or convert to .osm )

[Jaak Laineste]

> You can use the excellent Python script by Christopher Schmidt
> 
> http://crschmidt.net/blog/354/polyshp2osm/
> 

 I had the same problem, I had whole city data import and I messed with
different tools. Well, result is quite nice actually:
http://wiki.openstreetmap.org/wiki/Image:Tartu_osm_osmarenderer.png

Main problem was how to import tags properly (building addresses mainly).
With this Python script my impression was that it was made quite
specifically for the MassGIS, so it accepted some fields from it, but it
could not handle not my needed tags. So I tried and used successfully
another perl-based script shp2osm.pl, I think it was also in OSM Wiki (#
Copyright (c) 2006 Gabriel Ebner <ge at gabrielebner.at> updated in 2008 by
Tobias Wendorff <tobias.wendorff at uni-dortmund.de>). The only problem with
that was that I had to search required perl libraries for shapefile reading.
It would actually so nice to add all required dependencies, in their
compatible versions, to osm SVN also.

My whole procedure was as following. I wasted most time with tools and
methods which did not work, so hopefully this helps:

a) I needed to fix projections (data was in local projection) and data
classification and attributes
  1. I imported shapefiles to MapInfo Professional (commercial desktop GIS
soft for windows). It is not open source and far from free, but I have
worked with that. ArcGIS/View would do the same, and probably it could be
done with QGIS and free tools also.
  2. I modified classification: merged some files according to OSM tag
scheme. 
  3. Modified attribute table, so I have "column name" = "tag name", "data
value" = "tag value" schema.
  4. Exported data table with changing projection to LongLat WGS84 
  5. Converted it to shapefile

b) Converting to OSM XML file
 1. Installed perl, tried to execute shp2osm.pl, googled to find and install
required additional perl modules
 2. executed shp2osm.pl for all the shapefiles.
 3. opened osm files in "notepad++" and search-replaced tag names from
uppercase to lowercase in the XML. The shapefile table (actually plain old
DBF) has column names always in upper case. I'm not sure if this was really
necessary, maybe OSM sever would fix it for me anyway. Now I had proper OSM
files.

c) Uploaded data
 1. Downloaded and created manual backup for the area existing data, just in
case
 2. Opened OSM files in JOSM. I tried also some scripts to upload, but could
not get them working (also some dependencies what I was not able to find for
windows), but finally JOSM was just fine
 3. For street network I used validator plugin to fix topology. The original
data was totally missing topology, but JOSM validator fixed it quite well,
or at least pointed possible errors. Of course manual fixes was also needed,
especially with merging with existing roads, it was couple of nights manual
work (luckily there was not too much useful existing data). I wasted some
time trying to get GRASS properly working on my Windows to fix and check
topology beforehand, but could not get it running properly. I had one
working on Mac, but actual topology checking was not working there also. 
 4. Selected OSM file in JOSM, clicked Upload. I discovered that there is
actually no need to download existing data from server, and merge it with
copy-paste (I tried this too, but with it is pain with bigger datasets, and
turned out to be pointless also)
 5. After uploading I fixed additional topology (merged with existing data)
and tagging issues, with help of other community members. The upload of city
buildings (about 30.000 of them) with JOSM took like 3 nights with my home
500kbps or so upload. It had to be repeated several times, as in the first
time it was canceled and only nodes were there without connections. But next
time upload looked like continued nicely without repeating or duplicating
data.

Ps. Only remaining issue with the perl shp2osm.pl was that validator kept
complaining about duplicate nodes (polygon start/end node I guess) for some
of the shapes. I think it would be job of the converter to make sure that
particular elements do not have this kind of basic errors, and it is easy to
be fixed automatically.

/Jaakl