[Talk-us] My shape file import steps so far

Thu Sep 4 08:41:50 BST 2008

OK, I uploaded four versions of the python script with a readme to filebeam
http://filebeam.com/179cc73920afa0a187e76839e6ec9308
If your interested in converting shp to osm please take a look at that and
let me know if you get it working or not.
As they are based off a script by christopher schmidt, and a deravitive by
Ian Dees I'd love to hear what they think of the changes.
The scripts will not run on any other data besides what they are configured
for without editing.  Please look at the beginning comments of each.

I'd also like to hear what people think of the Seminole County data (using
shape_to_osm_Seminole_streets.py) compared to what is already there.  It
would still need work before submitting to osm, so please do not do that
yet.  Oneway streets and ramps direction, overpasses/bridges ect.

Dale

On Wed, Sep 3, 2008 at 7:20 PM, Dale Puch <dale.puch at gmail.com> wrote:

> OK I made some good strides on this.  I have what I think is a pretty good
> OSM file for all roads in Seminole County FL.
>
> Ian pointed me to shp_to_osm.py script which I had some trouble with.  I
> then went back and looked at the massgis_to_osm.py which it was based off.
> Looking thru it and trying it on the actual Mass data files also at the same
> location let me understand what the script was doing.  I ended up basing my
> work on the massgis copy as it did much of the (final) tagging in the script
> instead of afterwards by editing in JOSM.  That much data in JOSM makes it
> run pretty slow.
>
> I'll upload a copy of the script with FWTools hopefully in a "unzip and
> use" state so that getting python and the script working will no longer be
> any issue to using it.  It will still need editing for each shape file
> used!  The fields are different between GIS departments, and the script
> relies on the field names as well and what the data represents.
>
> That was the big advantage to Ian's version or the perl version I started
> with, it just reads all the field names and then used those as tags.  You
> were required to then edit the osm file in JOSM to change the shape fields
> into something meaningful to osm.  This is probably the better way to go for
> smaller files with limited data.  Stuff like POI's or trails ect.
>
> There is definitely a lot of progress that could be done with these scripts
> for someone that knows python or perl and working with shape file data.
> They could be automated a bit more than they are now.  Anyone out there? :)
>
>
> Anyhow now that I have a county's worth of road data...  What to do with
> it?  Ideally I would pull the old tiger data from the county (split at the
> county line somehow) and see what was left that was done manually (or edited
> by someone)  Compare what was left with the county data, and keep what was
> best.  Leaning towards the county data if all else is even, just for
> consistency sake.  And finally upload the data.  There would still remain
> the task of "attaching" roads that cross county lines.
>
> I still really want to work on merging data from separate shape files.
> There are separate files with road information line width, lanes, speed.
> Also files with street lights, exit info that could be applied to nodes.
> Has anyone done anything like this?  I saw all this in the Massgis data, but
> don't know if it was provided that way, or combined.  They also worked in
> smaller batches which would be nice.  So I imagine they worked thru some of
> these issues already.
>
> OK, more to come as I make sure this will all work for others.
>
> Dale
>
>
> On Tue, Sep 2, 2008 at 5:43 AM, Dale Puch <dale.puch at gmail.com> wrote:
>
>> I wanted to document what I tried, and get some feedback.  At the moment
>> I'm looking at http://www.seminolecountyfl.gov/gis/  Seminole county Fl
>> which may still need explicit response that the data is in the public domain
>> prior to uploading it.
>>
>> So far I have converted various downloaded shape files to WGS84 from their
>> respective state datums.  This took a while with me being a noob, and I
>> couldn't find anything that talked about what actually needed to be done.
>> So for anyone else out there, this was my method.
>>
>> Using ogr2ogr from FWTools2.1.0 http://fwtools.maptools.org/  Actually
>> the "Gis on a stick" (nice package)
>> http://www.archaeogeek.com/blog/portable-gis/
>> Most shape downloads include a *.prj file, so this makes it easy.  If your
>> missing one, try using another from the same source but make sure they use
>> the same settings.
>>
>> basic command:  ogr2ogr -t_srs WGS84 -s_srs "ESRI::in_shp.prj"
>> "out_shp.shp" "in_shp.shp"
>>
>> To automate it, a few dos commands to allow mass converting entire
>> directorie trees at once.  Starting at where the command was run from.
>> command:  FOR /R .\ %G IN (*.prj) DO
>> N:\usbgis\apps\FWTools2.1.0\bin\ogr2ogr -t_srs WGS84 -s_srs
>> "ESRI::%~dpnG.prj" "%~dpnG_out.shp" "%~dpnG.shp"
>>
>> This outputs with the same file name but with "_out" added
>> If you use this in a batch file, you will need to change the single % to
>> %%   A quirk of MS batch files.
>> you can also remove "/R .\" to only process files in the current
>> directory.
>>
>> Still to come is converting the WGS84 shape files into OSM format for
>> loading and review in JOSM followed by normal upload to the OSM database.
>> conversion script I found to start with...
>> http://raumplanung.tobwen.de/OSM/scripts/shp2osm_080814.pls
>> command:  FOR /R .\ %G IN (*_out.shp) DO shp2osm.pl "%~dpnG" >
>> "%~dpnG.osm"
>> Tests work well, and basic streets should be east to import.  Tags need to
>> be worked on though.  Like tiger (source:data_tag), and what tags should be
>> imported?  Do nodes need to be tagges as they were for the tiger data?  How
>> to pull attributes from other shp files and merge the data?  There are a few
>> places I think this would be the right thing to do.  Such as street lights
>> added to nodes, I saw some other data layers that might be of use in the
>> same way if they can be combined.
>>
>> One down side is the data points are higher density than they probably
>> need to be, so lots more data.  The ways by default are split at all
>> intersections, and there is cross street information (from, to) in each way
>> section.  Finally what to do with existing data such as the tiger stuff, and
>> especially user made stuff.
>>
>>
>>
>> Can anyone suggest a better method, or other comments?  import into
>> PostgreSQL perhaps?  Would this allow better review somehow, or processing
>> of smaller chunks of data?
>>
>>
>> --
>> Dale Puch
>>
>
>
>
> --
> Dale Puch
>

-- 
Dale Puch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-us/attachments/20080904/d4941c31/attachment.html>