[OSM-dev] planet_osm2txt with planet-060818?

[Andy Robinson]

Thanks for posting Dee.

Cheers,

Andy

Andy Robinson
Andy_J_Robinson at blueyonder.co.uk 

>-----Original Message-----
>From: dev-bounces at openstreetmap.org [mailto:dev-bounces at openstreetmap.org]
>On Behalf Of Dean Earley
>Sent: 21 August 2006 13:38
>To: 'Dev Openstreetmap'
>Subject: Re: [OSM-dev] planet_osm2txt with planet-060818?
>
>>>> Was anyone successfull with
>>>> ./utils/osm-pdf-atlas/planet_osm2txt.pl planet-060818-a.osm.bz2
>>>> Any hints on how to do this on a machine with only 2GB of Memory?
>>>>
>>>
>>> I guess the parsers will need to be rewritten to use SAX now rather
>>> than the DOM (read all and iterate) method. I happily parsed a
>>> 2.6GB planet.osm in about 2 minutes using MSXML (in VB)
>>
>> Would you mind sharing so more info on this approach. I'd be
>> interested in taking a look at some of the data in the planet file.
>
>Basically, it is an event based parser that keeps as little in memory as
>necessary.
>
>It notifies my code whenever a complete tag (start of and end of) has
>been found and any textual content.
>
>This is a sample of the VB code I used:
>
>Private Sub IVBSAXContentHandler_startElement(strNamespaceURI As String,
>strLocalName As String, strQName As String, ByVal attributes As
>MSXML2.IVBSAXAttributes)
>Dim ID As Long
>
>Dim Lat As Long
>Dim Lon As Long
>
>Dim ToNode As Long
>Dim FromNode As Long
>
>Static LastID As Long
>Static LastType As String
>
>   Select Case strLocalName
>   Case "osm"
>   Case "node"
>     ID = Val(attributes.getValueFromName("", "id"))
>     Lat = Val(attributes.getValueFromName("", "lat")) * 100000
>     Lon = Val(attributes.getValueFromName("", "lon")) * 100000
>
>     DB.Execute "INSERT INTO node (ID, Latitude, Longitude) VALUES (" &
>CStr(ID) & ", " & CStr(Lat) & ", " & CStr(Lon) & ");"
>
>     LastType = "node"
>     LastID = ID
>
>   Case "segment"
>     ID = Val(attributes.getValueFromName("", "id"))
>     ToNode = Val(attributes.getValueFromName("", "to"))
>     FromNode = Val(attributes.getValueFromName("", "from"))
>
>     DB.Execute "INSERT INTO segment (ID, 'To', 'From') VALUES (" &
>CStr(ID) & ", " & CStr(ToNode) & ", " & CStr(FromNode) & ");"
>
>     LastType = "segment"
>     LastID = ID
>
>   Case "way"
>     ID = Val(attributes.getValueFromName("", "id"))
>
>     DB.Execute "INSERT INTO way (ID) VALUES (" & CStr(ID) & ");"
>
>     LastType = "way"
>     LastID = ID
>
>   Case "seg"
>     DB.Execute "INSERT INTO waysegments (WayID, SegmentID) VALUES (" &
>CStr(LastID) & ", " & attributes.getValueFromName("", "id") & ");"
>
>   Case "tag"
>     DB.Execute "INSERT INTO tags (ID, Parent, Key, Value) VALUES (" &
>CStr(LastID) & ", """ & LastType & """, """ &
>Replace(attributes.getValueFromName("", "k"), """", "\""") & """, """ &
>Replace(attributes.getValueFromName("", "v"), """", "\""") & """);"
>
>   Case Else
>     'Stop on unrecognised tags
>     Stop
>   End Select
>End Sub
>
>It does require a syntactically valid XML file and unfortunately, this
>never completed as I ran out of disk space for the target db :)
>
>/me runs off to his flame proof bunker...
>
>--
>Dean Earley, Dee (dean at earlsoft.co.uk)
>
>irc:    irc://irc.blitzed.org/
>web:    http://personal.earlsoft.co.uk
>phone:  +44 (0)780 8369596
>
>_______________________________________________
>dev mailing list
>dev at openstreetmap.org
>http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev