[OSM-talk] OpenStreetMap Pledge

Wed May 31 00:03:31 BST 2006

On Tue, May 30, 2006 at 11:18:51PM +0100, Tom Carden wrote:
> On 30/05/06, Christopher Schmidt <crschmidt at crschmidt.net> wrote:
> >On Tue, May 30, 2006 at 10:36:51PM +0100, Tom Carden wrote:
> >>
> >What I believe is more likely is that mapserver was being used in a way
> >that it was not as efficient as possible, and therefore, was acting more
> >slowly than one would expect.
> 
> And also because the current database is set up to be read/write and
> record all changes.

> The reason openstreetmap.org is slower is because it takes all the
> changes into account and shows the current data (which Nick's code
> doesn't do, and which your Mapserver implementation couldn't and
> wouldn't need to do).  I believe it will be possible in the future to

"Couldn't" isn't neccesarily true. I will go with "Wouldn't, for the
time being, or in the near future."

At the moment, with how OpenStreetMap works, 
changes don't get displayed immediately anyway: in the current way
the code works, they're displayed no more than every 2 days, because
that's how long they live in the cache. The idea of a rebuilt nightly
(or, depending on the load this takes off the database, possibly more
often) cache means that that you're not losing anything you aren't
already losing with the current code.

The idea and purpose of the nightly dump is to make data that is more
accurate than the current way OSm does things (which is 48 hour
timeouts), faster, and taking the load off the database server so that
it can go back to doing what databases do best: writing things. Using a
database to select from a mostly-stable set of data for every query is
so wasteful that I can't help but think that moving to a different
system might help out all aspects of OSM. 

> >In the past month I've been working with
> >OSM data, I've watched tile rendering times for my OSM based maps
> >decrease fivefold to twentyfold, as I learned more about how to make
> >things fast.
> 
> Sounds great! Please share!

Google "Tiled shapefiles", "GRASS", "Polyline Simplification". All of
those are a great step to taking what OSM is now -- a set of 500,000
segments -- and turning it into something more sane: 90,000 polylines --
and simplifying that such that you get an accurate picture at a given
zoom level without needing to draw every curve in the road: WHen I'm
viewing all of the UK, I don't need to know that Outer Circle is made up
of 17 line segments: it's likely that a simplification as a square will
do. Tiled shapefiles offer something that might not otherwise be
available: an index into the shapefile, to select the bounding box
quickly.

The work that has been done has been a joint effort between myself,
Schuyler Erle, and Jo Walsh. There's no documentation/release because at
the moment, it's hard to create a timely, interesting, accurate product.
I'm hoping to get the motivation to change that with a good dataset at
hand.

> >In other words, once I *have* an implementation, I'd be glad to run it
> >on OSM's servers -- if it was desired. Until I do, there's no point in
> >me wasting anyone's time with a solution that's already been rejected.
> 
> Is this like, "I have discovered a way to render OpenStreetMap data
> quickly and correctly using Mapserver, but the margin is too small to
> write it here"?

*shrug* I'm already doing it, but it's not as quick as I'd like, but
more importantly, it's frustrating to always have to be a month behind
the curve in creating styles that match the current styles for mapping
out roads. The Isle of Wight workshop gave a wonderfully rich set of
data: But I've never been able to get the data out of the API (always
times out) and it's not in the planet.osm dump that I can get. So I have
to wait until the next time someone creates a planet.osm dump to get the
data out to create appropriate styling.

With this, it's really discouraging: I can't go in, edit OSM, and wait
24 hours to see my updates in a pretty interface: I have to wait a
month. (Again, impatience.) I think that a higher level of feedback to
users will lead to more editors, and more editors leads to better data
leads to better maps.

Despite all my complaining, I would like nothing more than to see
OpenStreetMap succeed wildly. However, I feel very much like I'm pushing
against a brick wall every time I try to help. I'm sure this is simply
differences of opinion in the way things should be done, so I'm willing
to go into my corner and work on things until I come up with something
that OSM can use.

I've already created a visual display of OSM segments -- loads quick,
even at the worldwide level. (The entire UK did take 4 seconds to
render, I'll admit.)

http://crschmidt.net/mapping/openlayers/osm

IT's built off the only planet.osm data I could find -- from beginning
of April, all the mirrors have  old data or weren't responding -- but
even so, I think it's pretty cool.

But in order to create maps that actually style things accurately --
changing colors for highways and the like -- I need data that's
accurately tagged with attributes, and the April planet.osm data has a
tiny fraction of the tagged segments/ways that the latest data does.

It's really hard to motivate yourself to accurately display 2 month old
data.

-- 
Christopher Schmidt
Web Developer