[Talk-us] My shape file import steps so far

[Dale Puch]

OK I made some good strides on this.  I have what I think is a pretty good
OSM file for all roads in Seminole County FL.

Ian pointed me to shp_to_osm.py script which I had some trouble with.  I
then went back and looked at the massgis_to_osm.py which it was based off.
Looking thru it and trying it on the actual Mass data files also at the same
location let me understand what the script was doing.  I ended up basing my
work on the massgis copy as it did much of the (final) tagging in the script
instead of afterwards by editing in JOSM.  That much data in JOSM makes it
run pretty slow.

I'll upload a copy of the script with FWTools hopefully in a "unzip and use"
state so that getting python and the script working will no longer be any
issue to using it.  It will still need editing for each shape file used!
The fields are different between GIS departments, and the script relies on
the field names as well and what the data represents.

That was the big advantage to Ian's version or the perl version I started
with, it just reads all the field names and then used those as tags.  You
were required to then edit the osm file in JOSM to change the shape fields
into something meaningful to osm.  This is probably the better way to go for
smaller files with limited data.  Stuff like POI's or trails ect.

There is definitely a lot of progress that could be done with these scripts
for someone that knows python or perl and working with shape file data.
They could be automated a bit more than they are now.  Anyone out there? :)


Anyhow now that I have a county's worth of road data...  What to do with
it?  Ideally I would pull the old tiger data from the county (split at the
county line somehow) and see what was left that was done manually (or edited
by someone)  Compare what was left with the county data, and keep what was
best.  Leaning towards the county data if all else is even, just for
consistency sake.  And finally upload the data.  There would still remain
the task of "attaching" roads that cross county lines.

I still really want to work on merging data from separate shape files.
There are separate files with road information line width, lanes, speed.
Also files with street lights, exit info that could be applied to nodes.
Has anyone done anything like this?  I saw all this in the Massgis data, but
don't know if it was provided that way, or combined.  They also worked in
smaller batches which would be nice.  So I imagine they worked thru some of
these issues already.

OK, more to come as I make sure this will all work for others.

Dale


On Tue, Sep 2, 2008 at 5:43 AM, Dale Puch <dale.puch at gmail.com> wrote:

> I wanted to document what I tried, and get some feedback.  At the moment
> I'm looking at http://www.seminolecountyfl.gov/gis/  Seminole county Fl
> which may still need explicit response that the data is in the public domain
> prior to uploading it.
>
> So far I have converted various downloaded shape files to WGS84 from their
> respective state datums.  This took a while with me being a noob, and I
> couldn't find anything that talked about what actually needed to be done.
> So for anyone else out there, this was my method.
>
> Using ogr2ogr from FWTools2.1.0 http://fwtools.maptools.org/  Actually the
> "Gis on a stick" (nice package)
> http://www.archaeogeek.com/blog/portable-gis/
> Most shape downloads include a *.prj file, so this makes it easy.  If your
> missing one, try using another from the same source but make sure they use
> the same settings.
>
> basic command:  ogr2ogr -t_srs WGS84 -s_srs "ESRI::in_shp.prj"
> "out_shp.shp" "in_shp.shp"
>
> To automate it, a few dos commands to allow mass converting entire
> directorie trees at once.  Starting at where the command was run from.
> command:  FOR /R .\ %G IN (*.prj) DO
> N:\usbgis\apps\FWTools2.1.0\bin\ogr2ogr -t_srs WGS84 -s_srs
> "ESRI::%~dpnG.prj" "%~dpnG_out.shp" "%~dpnG.shp"
>
> This outputs with the same file name but with "_out" added
> If you use this in a batch file, you will need to change the single % to
> %%   A quirk of MS batch files.
> you can also remove "/R .\" to only process files in the current directory.
>
> Still to come is converting the WGS84 shape files into OSM format for
> loading and review in JOSM followed by normal upload to the OSM database.
> conversion script I found to start with...
> http://raumplanung.tobwen.de/OSM/scripts/shp2osm_080814.pls
> command:  FOR /R .\ %G IN (*_out.shp) DO shp2osm.pl "%~dpnG" > "%~dpnG.osm"
> Tests work well, and basic streets should be east to import.  Tags need to
> be worked on though.  Like tiger (source:data_tag), and what tags should be
> imported?  Do nodes need to be tagges as they were for the tiger data?  How
> to pull attributes from other shp files and merge the data?  There are a few
> places I think this would be the right thing to do.  Such as street lights
> added to nodes, I saw some other data layers that might be of use in the
> same way if they can be combined.
>
> One down side is the data points are higher density than they probably need
> to be, so lots more data.  The ways by default are split at all
> intersections, and there is cross street information (from, to) in each way
> section.  Finally what to do with existing data such as the tiger stuff, and
> especially user made stuff.
>
>
>
> Can anyone suggest a better method, or other comments?  import into
> PostgreSQL perhaps?  Would this allow better review somehow, or processing
> of smaller chunks of data?
>
>
> --
> Dale Puch
>



-- 
Dale Puch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-us/attachments/20080903/4a514d07/attachment.html>