[OSM-dev] planet.openstreetmap.org

[Joerg Ostertag (OSM Munich/Germany)]

On Friday 18 August 2006 13:40, Nicola Ranaldo wrote:
> On Friday 18 August 2006 13:07, Jonas Svensson wrote:
> > On Fri, 18 Aug 2006, Nicola Ranaldo wrote:
> > > The following part of last planet.osm seems to be not valid:
> > >
> > > <node id="543408" lat="51.2714" lon="7.13737"
> >
> > There seems to be about 490 UTF-8 errors in this dump too.
>
> Umh... this is annoying, i spent about 3 hours with valgrind, my problem
> the expat library i use to parse data, and when it encounters that it
> buffers all the subsequent line in ram, ignoring the other elements, on a
> big file like the last dump my system swaps and goes out of memory :)
>
> I do not know if i have to double check expat docs...
> Has anyone the same problem?

I have a sed script to filter the file in advance. SO no broken UFT-8 appears. 
For now these are not interesting to me. So it works.

The sed part is quick and dirty, but it seems to work.

-

Joerg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix.sh
Type: application/x-shellscript
Size: 3560 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20060818/0615d372/attachment.bin>