[Routing] export osm to use it in pgrouting

[Jingmin Chen]

Hello Markus,

I have a simple solution for OSM4routing with large data.
In osm4routing.py, I add a few lines of code such that it commits a transaction to the database  every 1000 records.
It solves my problem when I deal with Switzerland data (well, a relatively small country)
Here is my code for step 3. 

    print "Step 3: saving the edges"
    edges = p.get_edges()
    print "get all edeges"
    count = 0
    if output == "csv":
        e = open(edges_name + '.csv', 'w')
        e.write('"edge_id","source","target","length","car","car reverse","bike","bike reverse","foot","WKT"\n')
    for edge in edges:
        if output == "csv":
            e.write('{0},{1},{2},{3},{4},{5},{6},{7},{8},LINESTRING({9})\n'.format(edge.edge_id, edge.osm_id, edge.source, edge.target, edge.length, edge.car, edge.car_d, edge.bike, edge.bike_d, edge.foot, edge.geom))
        else:
            session.add(Edge(edge.edge_id, edge.osm_id, edge.source, edge.target, edge.length, edge.car, edge.car_d, edge.bike, edge.bike_d, edge.foot, edge.geom, spatial=spatial))
        count += 1
        if count%1000==0:
            session.commit()
        if count%10000==0:
            print count
    if output == "csv":
        e.close()
    else:
        session.commit()
    print "  Wrote {0} edges\n".format(count)


Best regards.

Jingmin Chen

On Aug 31, 2011, at 5:53 PM, Markus wrote:

> 
> 
> Von: Daniel Kastl <daniel at georepublic.de>
> An: routing at openstreetmap.org
> Gesendet: 15:18 Dienstag, 30.August 2011 
> Betreff: Re: [Routing] export osm to use it in pgrouting
> 
> osm2pgrouting has problems with large amount of data.
> It depends how much memory you have available.
> 
> I am using 2 GB of memory. Not enough??
> 
> 
> This could be one reason.
> Can you make sure that you created a database and loaded all required functions?
> You can also create a routing_template like here:
> https://github.com/pgRouting/workshop/blob/master/bin/create_templates.sh
> 
> I did all what you wrote, but I obtained the same problem.
> 
> 
> Now I tried osm4routing: launching:
> 
> Postgis output option:
> osm4routing -o postgresql://postgres@localhost/routing -n italy_nodes -e italy_edges -s /opt/tmp/italy.osm
> 
> this brings an error after parsing while trying to save in the db
> 
> Csv output option:
> osm4routing -o csv -n italy_nodes -e italy_edges -s /opt/tmp/italy.osm
> 
> this save the nodes in the csv file but while saving the edges I obtain:
> Read 33314731 nodes and 2849260 ways
> 
> Step 2: saving the nodes
>   Wrote 1750193 nodes
> 
> Step 3: saving the edges
> terminate called after throwing an instance of 'std::bad_alloc'
>   what():  std::bad_alloc
> Aborted
> 
> 
> it looks like that both of this tools have problems with large data.
> 
> 
> 
> _______________________________________________
> Routing mailing list
> Routing at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/routing

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/routing/attachments/20110831/c310b2ee/attachment.html>