[OSM-dev] UTF-8 problems in informationfreeway?

Stefan Baebler stefan.baebler at gmail.com
Mon Dec 17 10:01:15 GMT 2007


Ok, I'll try this in the evening (CET):
- make a diff file between older slovenia-071128.osm.bz2 (without
Moravče) and a newer one slovenia-071205.osm.bz2 with it, to see the
actual diff
- load old slovenia-071128.osm.bz2 into db, then apply the diff & see
what happens
Something else?

If someone wants to see results earlier, there are the osm files:
http://osm.baebler.net/data/

Maybe it is also worth inspecting the
http://planet.openstreetmap.org/planet-071128-071205.diff.xml.bz2
or whatever diff (hourly?) was used to get changes into informationfreeway db.

Stefan

On Dec 17, 2007 10:21 AM, 80n <80n80n at gmail.com> wrote:
> Brett
> Yes, it's probably something like that.  All I can say for sure is that the
> node in planet.osm looks different to the same one in an Osmosis diff file.
>
> If we could identify how it is encoded differently then maybe I could
> compensate for it on import into Osmxapi, but it would be better to fix the
> problem at source - wherever that it.
>
> Anyway, there's no rush to deal with it at the moment.
> 80n
>
>
>
> On Dec 17, 2007 6:05 AM, Brett Henderson <brett at bretth.com> wrote:
> > It warms my heart to return from leave to discover new osmosis utf8
> > problems, I missed those little guys ;-)
> >
> > I'll check it out.  Might take me a few days though because I'm a bit
> > overwhelmed with email and Christmas at the moment ...  Strictly
> > speaking I suspect this is not truly a bug but yet another artefact of
> > the database encoding issues, it may not be easy to nail.
> >
> >
> >
> >
> > Stefan Baebler wrote:
> > > Another artifact of similar utf problem can be seen at yesterday's
> > > lowzoom(!) tile:
> > > http://tah.openstreetmap.org/Tiles/info.php?x=1107&y=727&z=11&layer=tile
> > > Mengeš ("š" is ok)
> > > Domžale ("ž" is ok)
> > > Moravče ("č" turned into "Ä ")
> > >
> > > On zoom 12 and higher "č" in Moravče is ok:
> > >
> http://tah.openstreetmap.org/Tiles/info.php?x=4431&y=2909&z=13&layer=tile
> > >
> > > There definitely is a problem _somewhere_.
> > >
> > > In today's and last week's dump node is ok (extract made with osmosis!):
> > >   <node id="29161753" timestamp="2007-12-02T08:52:13Z"
> > > lat="46.1356895" lon="14.7445634">
> > >     <tag k="created_by" v="JOSM"/>
> > >     <tag k="name" v="Moravče"/>
> > >     <tag k="place" v="town"/>
> > >   </node>
> > > (this xml snippet is an extract of a planet file, done with osmosis
> > > for local archive: http://osm.baebler.net/data/ )
> > >
> > > Osmosis seems to handle that in files(!) just fine, but osmxapi gives it
> wrong:
> > >
> > >
> http://www.informationfreeway.org/api/0.5/node%5bplace=town%5d%5bbbox=14.5,46.1,14.8,46.2%5d
> > > either there is a bug in osmxapi or during the import into its db.
> > >
> > > hope it helps tracking it down.
> > >
> > > greets,
> > > Štefan
> > >
> > > On Dec 8, 2007 12:54 AM, Frederik Ramm < frederik at remote.org> wrote:
> > >
> > >> Hi,
> > >>
> > >>
> > >>> This appears to be an osmosis problem.
> > >>> A recent planet contains the following:
> > >>>
> > >> [...]
> > >>
> > >> I concur; the latest daily diff before the Dec06 planet file had UTF-8
> > >> problems as well but the affected objects were represented ok in the
> > >> planet file.
> > >>
> > >> Bye
> > >> Frederik
> > >>
> > >> _______________________________________________
> > >>
> > >> dev mailing list
> > >> dev at openstreetmap.org
> > >> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
> > >>
> > >>
> > > _______________________________________________
> > > dev mailing list
> > > dev at openstreetmap.org
> > > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
> > >
> >
> >
> > _______________________________________________
> > dev mailing list
> > dev at openstreetmap.org
> > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
> >
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
>
>


More information about the dev mailing list