[OSRM-talk] Navteq (esri .map) to osrm problems

[Alan Grover]

(Rather than start with a plea for help, I'd _really_ thank you for osrm.)

I'm using osrm for time/distance matrices, and geometry (and
turn-by-turn in the near future). I've been running it with the
berlin-latest data from openstreetmap for testing/development. That's
been working well, and impressively.

Of course, I need to use Navteq U.S. and Canada for production. But, I
can't figure out how to get the data into osrm (yea, I know this will
take a gazillion cpu cycles, and a lot of ram to do the conversion.)

The first question is: is this too much of a task for someone (me) who
is not a gis expert? Do you know who I can hire to help me with this.

The second question is: can you help me figure out the steps to get
navteq data into osrm?

I have only a spotty understanding of the data/concepts for esri & osrm,
so I can't figure out what is wrong:

1. The navteq data was being used in esri. Our gis specialist (doesn't
know openstreetmap) exported it in ".map" format. Here's the cut-down
files I start with (just Delaware):

ls -l map-data/Delaware.map
-rw-rw---- 1 me me 36617162 Jan 19  2015 Streets.DAT
-rw-rw---- 1 me me   349536 Jan 19  2015 Streets.ID
-rw-rw---- 1 me me 21741056 Jan 19  2015 Streets.IND
-rw-rw---- 1 me me  7585280 Jan 19  2015 Streets.MAP
-rw-rw---- 1 me me     3086 Jan 19  2015 Streets.TAB
-rw-rw---- 1 me me 10654217 Jan 19  2015 Zlevels.DAT
-rw-rw---- 1 me me  1469516 Jan 19  2015 Zlevels.ID
-rw-rw---- 1 me me  9404416 Jan 19  2015 Zlevels.IND
-rw-rw---- 1 me me  6843904 Jan 19  2015 Zlevels.MAP
-rw-rw---- 1 me me      305 Jan 19  2015 Zlevels.TAB

2. I choose ogr2osm.py (https://github.com/pnorman/ogr2osm.git) to do
the conversion to osm. I choose that because it seemed to be relatively
up-to-date, and one-step (compared to -> gml -> osm). Maybe I should
have used a different process?

python map-data/ogr2osm.py -d -o map-data/Delaware.map.osm/Delaware.osm
map-data/Delaware.map/

running with lxml.etree
Preparing to convert 'map-data/Delaware.map/' to
'map-data/Delaware.map.osm/Delaware.osm'.
Will try to detect projection from source metadata, or fall back to
EPSG:4326
Using default translations
Using default filterLayer
Using default filterFeature
Using default filterTags
Using default filterFeaturePost
Using default preOutputTransform
Parsing data
Detected projection metadata:
GEOGCS["unnamed",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563],
        TOWGS84[0,0,0,0,0,0,0]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433]]
Detected projection metadata:
GEOGCS["unnamed",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563],
        TOWGS84[0,0,0,0,0,0,0]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433]]
Merging points
Making list
Checking list
Merging duplicate points in ways
Outputting XML

-rw-rw---- 1 me me 325449558 Dec 16 12:09 Delaware.osm

3. It seems like osrm-extract will accept .osm files, so I used that.

The profile.lua is osrm-backend/profiles/car.lua.

osrm-backend is https://github.com/Project-OSRM/osrm-backend.git,
	branch develop
	last commit: Date:   Fri Nov 20 19:52:22 2015 +0100
	compiled on ubuntu 14.04 as per instructions.
(nb, the osrm stuff worked fine "extracting" and "preparing" the
berlin-latest data).

I was using the develop branch because it seemed to have more recent
fixes. Perhaps I should be on master for stability? (yes, this will be
used for production processes, periodically).

So, the obvious problem happens here:

osrm-extract map-data/Delaware.map.osm/Delaware.osm

[info] Input file: Delaware.osm
[info] Profile: profile.lua
[info] Threads: 4
[info] Using script profile.lua
[STXXL-MSG] STXXL v1.3.1 (release)
[STXXL-ERRMSG] Warning: no config file found.
[STXXL-ERRMSG] Using default disk configuration.
[STXXL-MSG] 1 disks are allocated, total space: 1000 MiB
[info] Parsing in progress..
[info] input file generated by uvmogr2osm
[info] timestamp: n/a
[info] Using turn restrictions
[info] Found 3 exceptions to turn restrictions:
[info]   motorcar
[info]   motor_vehicle
[info]   vehicle
[info] Parsing finished after 5.12737 seconds
[info] Raw input contains 260230 nodes, 87384 ways, and 0 relations, and
0 unknown entities
[warn] The input data is empty, exiting.
[STXXL-ERRMSG] Removing disk file created from default configuration:
/var/tmp/stxxl

And no new (data) files. I notice that the "relations" is empty, but I
don't know if that is significant.

-- 
Alan Grover
agrover at c2logix.com
+1.734.476.0969