[OSM-talk] Tiles at home Update

80n 80n80n at gmail.com
Thu Mar 22 10:41:04 GMT 2007


I'm currently looking at Frollo to see if there is something wrong
with it.  For some cases it seems to be using significantly more
resources (CPU and RAM) than I would have expected to be needed and
more than I ever saw during testing.

There are a number of possible reasons that I'm looking into at the moment:
1) xmlstarlet might not be optimising tail-recursion properly
2) the frollo xsl template might not be coded correctly to allow
tail-recursion optimisation
3) xmlstarlet on some platforms (or some versions) behaves incorrectly
or has a bug
4) there may be an xpath that would benefit from an index (there's
definitely one case in frollo1, not sure about frollo2).
5) there could be a badly written xpath expression
6) other xsl processors (eg xalan) seem to run frollo faster - perhaps
xmlstarlet is not the best choice.

Some of the things that frollo does should actually help Osmarender to
run faster.  My expectation (and goal) is that frollo + (osmarender *
6)  should be faster than (osmarender * 6).  Osmarender runs 6 times,
once for each zoom level, so doing some things once in frollo should
have a 6x saving vs doing the same thing in osmarender.

Right now frollo isn't performing as well as it theoretically should.
When it does the use of frollo should be a non-issue (unless edmix up
tstree nam arees desirable ;)

80n

On 3/22/07, Dirk-Lüder Kreie <osm-list at deelkar.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Dave schrieb:
> > matthew-osm at newtoncomputing.co.uk> wrote:
> >> On the other hand, maybe the definition actually is
> >> "ordered" and I missed that bit ;-).
> >
> > They are ordered, as in the API/DB will return them in the same order they
> > were uploaded. That's in the spec.
> > The only problem is that there is no semantics defined as to what that
> > ordering should be -- it's entirely up to the uploaders. So no, they're not
> > errors as such, other than they don't render nicely without preprocessing,
> > and there seems to be a general consensus on what constitutes a nice way
> > for these purposes, and they aren't it.
>
>
>
> > Of course, in a future version of the OSM data model it's possible this
> > might change, at which time they may become errors, but until then
> > renderers/editors etc will just have to cope (or not cope, with the
> > intention of making users change things manually, assuming they can
> > actually
> > figure out what's wrong).
> >
> > Personally I think coping is the best way to go... OSM data will always
> > be a
> > little fuzzy round the edges, and not rendering a mess makes sense to me.
>
> I'm not sure about the percentage worldwide, but I know for at least 2
> German Cities (Karlsruhe and Bremen) that there at least the Frollo
> preprocessor is a giant waste of time and resources, because there are
> next to no unordered ways there. However the overhead introduced by
> Frollo brings it back to rendering times before any of the optimizations
> thought out by Ojw and others (including me). Besides, the preprocessor
> shuts out a whole lot of renderers that simply can't render any tiles
> that are a little more complete, London for instance is barely updated
> anymore. In IRC there have been several mentions of tile areas that
> simply don't render anymore on some clients because the preprocessor
> just fails.
>
> While I agree that it is better to have nice maps I still think the
> rendering preprocessor is the worst place to "correct" unordered ways.
>
> I was always under the impression that ways should be well ordered,
> preferably non-branching and contiguous, because that way it's the
> easiest to actually *do* things with the data (not just rendering).
>
> So if nobody cares what order the segments in a way are but the
> renderers (there are other uses that would benefit from ordered ways)
> then why not create the ways ordered? JOSM for instance has every tool
> needed to help this along.
>
> Concerning the "fuzzy around the edges" part nobody hinders anyone
> running frollo on a piece of data and uploading the result back to the
> db, which would fix that area not only for the one rendering but for
> other uses and most of the time even future renders of that area, too.
> Why does this have to be done again and again *every* *single* *time*?
>
> And no, I don't like the idea of having the Frollo preprocessor upload
> it's changes under my username automatically within the tiles at home
> project, for reasons which should be quite obvious.
>
> To summarize:
>
> - - Frollo shuts out renderers from tiles at home
> - - Frollo takes an insane amount of resources (RAM and time) for what it
>   does
> - - Frollo just fixes the data locally for *one* rendering job, no other
>   benefit comes from it
> - - The resources needed to process densely mapped tiles exceed those
>   available on some tiles at home renderers
> - - The data can be fixed in the main db easily
> - - Errors on the osmarender (t at h) layer help to locate errors, and these
>   are easily corrected and the affected tiles can easily be re-rendererd
>
> So why again, are we using it in tiles at home?
>
> Because it renders nicer maps. Or does it?
>
> I'm not at all against some sort of preprocessing, Frollo seems to do a
> good job at what it does for single renderings, so I'm not generally
> against this script, just that it doesn't make any sense to me using it
> in t at h, considering the cost.
>
> Dirk-Lüder "Deelkar" Kreie
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFGAgkYFUbODdpRVDwRAqYxAJ9TkkGkitmvLjLQP7vUC8aV80vKRACgkOFY
> v9n/38Hz04QeeWacg8mYfHY=
> =H8Xq
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> talk mailing list
> talk at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
>




More information about the talk mailing list