[OSM-dev] Re: Re: [OSM-talk] planet.dump

[Erik Johansson]

On 8/2/06, Jonas Svensson <jonass at lysator.liu.se> wrote:
> On Wed, 2 Aug 2006, SteveC wrote:
>
> > > I believe that the code that generates the dumps is
> > >     http://svn.openstreetmap.org/utils/planet.osm/planet.rb
> > >
> > > I'm sure that Steve would appreciate patches to make it better :)
> >
> > yes please
>
> I do not know anything about ruby at the moment. But one thing to check is
> any changes in the code or the environment between the may and july dump.
> The may dump seems to be properly (html-entity) encoded, while the july is
> lacking. That is one problem, the other is the data in the database. The
> july dump suggests (at least to me) that the database contains a mixture
> of charsets. Maybe we also have to make sure any data entered into the
> database is either converted to utf-8 or at least tagged with the proper
> coding.

The past dumps used an XML library and now it's printed with println.
The section that needs improvement is:

			v1 = v.gsub(/[']/,"'") # escape quotes
			v2 = v1.gsub(/</,"<") # escape <
			v3 = v2.gsub(/>/,">") # escape >
			puts "<tag k='#{k}' v='#{v3}' />"


I think you can manage that. ;-)