[Imports] shp attributes

Richard Weait richard at weait.com
Tue Nov 10 01:00:39 GMT 2009


On Mon, Nov 9, 2009 at 7:28 PM, Simone Cortesi <simone at cortesi.com> wrote:
> hi,
>
> just a short question to keep the ball rolling: i'm getting my hands
> on some new good data for my region. they do have good metadata and
> the feature attributes of the shape files are really good, and seem to
> natively cover the extent of osm map_feature.
>
> due to the fact that there are several classes, themes and types of
> data, my list of shp-to-osm rules is getting bigger and bigger even if
> i merely begun working on this new set of data.
>
> any idea on how to organize this on the web in order to better manage
> all of this information?
>
> I need a common source, i.e. a single location, from wich to extract a
> rules.txt file that will then later will be passed to shp-to-osm.

Dear Simone,

The canvec data is provided in many separate shp files, typically one
feature type per file.  So separate rule files have worked pretty
well.

I've attached the script I've been using to bulk-convert collections
of shp files.  It checks a config file for the feature types I want to
convert, then converts each previously unconverted shape file.  It
could easily use one common rule file rather than the separate rule
files per feature.

Best way to maintain your rule files on the web?  I don't know.  How
about github?

Best regards,
Richard
-------------- next part --------------
#!/bin/bash 
# create a group of .osm files from a group of CanVec zip files
#

# set directories for your system
# comment entity lines in .conf to supress conversion

  export USER=`whoami`
  export HOME=/home/$USER
  export DATA=$HOME/canvec-data
  export SHPZIP=$DATA/shp-zip
  export SHP=$DATA/shp
  export OSM=$DATA/osm
  export SHPLOCK=$DATA/shp2osm.lock
  export CANVEC2OSMDIR=$HOME/canvec-shp2osm
  export RULESDIR=$CANVEC2OSMDIR/rulesTXT
  export CONFIG=$CANVEC2OSMDIR/canvec-shp2osm.conf
  export CANVECBIN=$CANVEC2OSMDIR/shp-to-osm-0.7.2/shp-to-osm-0.7.2-jar-with-dependencies.jar
  export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.16/bin

  if [ ! -f $SHPLOCK ]; then
    touch $SHPLOCK
    # for each canvec zip file
    for file in `ls --format=single-column $SHPZIP/canvec_??????_shp.zip`
    do
      quad=`basename $file | sed 's/canvec_\([0-9][0-9][0-9][a-z][0-9][0-9]\)_shp.zip/\1/' `
      if [ -d $SHP/$quad ]; then
        echo "[  skipping  ] directory $SHP/$quad exists"
      else
        # create a directory for unzipped data and for converted .osm data
        echo "[ create dir ] $SHP/$quad"
        mkdir -p $SHP/$quad
        mkdir -p $OSM/$quad
        # unzip to the new directory
        echo "[    unzip   ] $file into $SHP/$quad"
        unzip -q $file -d $SHP/$quad
        # for each uncommented entity code in config 
        egrep -v '(^#|^;|^\w*$)' $CONFIG | sed 's/;.*$//;s/#.*$//' | ( while read line
        do
          echo "[  request   ] $line entities" 
          # find shapefiles for this entity
          for shpfile in `ls --format=single-column $SHP/$quad/*$line*.shp`
          do
            if [ "$shpfile" == "" ]; then
              echo "[ not found  ] line: $line shapefile: $shpfile"
            else
              echo "[   found    ] line: $line shapefile: $shpfile"
              # make the filename managable
              shpbase=`basename $shpfile | cut -d "_" -f 5`
              # find matching rule file
              rulecount=`ls --format=single-column $RULESDIR/$line*RULES.txt | grep -c RULES`
              if [ $rulecount == 0 ]; then
                echo "[    error   ] no rule file found for $line"
              else
                # use the first matching rule file 
                rules=`ls --format=single-column $RULESDIR/$line*RULES.txt | head -n 1`
                if [ $rulecount != 1 ]; then
                  echo "[  warning   ] using $rules of multiple matching rule files"
                fi
                # run it through shp-to-osm
                echo "[shape to osm] $shpfile rules $rules output-osm $OSM/$quad/$shpbase.osm"
                # make the filename managable
                shpbase=`basename $rules | sed "s/RULES.txt/-/"`
                # run it through shp-to-osm
                echo "[shape to osm] $shpfile rules $RULES output-osm $OSM/$quad/$shpbase.osm"
#                $JAVA_HOME/java -Xmx512M -cp $CANVECBIN Main --shapefile $shpfile --rulesfile $rules --osmfile $shpbase --outdir $OSM/$quad --maxnodes 5000 --outputFormat osm -t
                $JAVA_HOME/java -Xmx512M -cp $CANVECBIN Main --shapefile $shpfile --rulesfile old-rules.txt --osmfile $shpbase --outdir $OSM/$quad --outputFormat osm -t
              fi 
            fi  
          done 
        done ) 
      fi
    done
    rm $SHPLOCK
  else
    echo "Locked"
    exit 1;
  fi


More information about the Imports mailing list