[OSM-talk] talk Digest, Vol 33, Issue 11

Fri May 4 02:29:07 BST 2007

I am a pro. at mapping even, so you can take my kvetching seriously (if you 
want) or not.
I've been using the interface and listening to the discussions for a while 
this is an observation.
I think my reactions are not that uncommon for people who are editing, so it 
may have some use.
Lewis
----- Original Message ----- 
From: <talk-request at openstreetmap.org>
To: <talk at openstreetmap.org>
Sent: Thursday, May 03, 2007 5:39 PM
Subject: talk Digest, Vol 33, Issue 11


> Send talk mailing list submissions to
> talk at openstreetmap.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
> or, via email, send a message with subject or body 'help' to
> talk-request at openstreetmap.org
>
> You can reach the person managing the list at
> talk-owner at openstreetmap.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of talk digest..."
>
>
> Today's Topics:
>
>   1. Re: riverbanks, way, segs, corrruption, user experience,
>      traction. (Robert Hart)
>   2. Re: live rendering (Robert (Jamie) Munro)
>   3. Karlsruhe area (was Black Forest) (Nick Whitelegg)
>   4. Re: live rendering (Schuyler Erle)
>   5. Re: live rendering (Frederik Ramm)
>   6. Re: riverbanks, way, segs, corrruption, user experience,
>      traction. (OJW)
>   7. Tourlaville (David Earl)
>   8. Re: live rendering (Martijn van Oosterhout)
>   9. Richmond on Thames Mapping Party - Accommodation (80n)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 3 May 2007 16:29:20 +0100
> From: "Robert Hart" <bathterror at gmail.com>
> Subject: Re: [OSM-talk] riverbanks, way, segs, corrruption, user
> experience, traction.
> To: lewispusey <lewispusey at earthlink.net>
> Cc: talk at openstreetmap.org
> Message-ID:
> <427adff80705030829k16a536d5kf5f40f068dc473de at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> On 03/05/07, lewispusey <lewispusey at earthlink.net> wrote:
>> Deleting and redrawing the ways will require days more of work and since 
>> I
>> redrew them once; that was several more days, and the original drawing 
>> then
>> alsotook days. I think this points to a problem, losing original 
>> information
>> and time consuming redraws for no real reason (other than the first pass 
>> was
>> not right?).
>
> it really can't be all that bad. You delete the ways, but still have
> all the segments left over. A few hours perhaps for a really long
> river - but why were you trying to do all that at once without testing
> things out on a small section? If you try and run before you can walk,
> then problems are fairly likely.
>
> There is always going to be a learning curve with something like this
> - so be patient. You'll be mapping like a pro in no time.
>
>>      I just spent many hours redrawing a large lake that was duplicated 
>> or
>> broken on upload repeatedly. For something that I do for free on my spare
>> time that level of frustration is not realy acceptable. I'm not saying 
>> this
>> to aggravate anyone, simply: if you want non-commercial contribution
>> something needs to be a lot more editor friendly soon from an individual
>> editing perspective. Trial and error (only partly my problem as I did do 
>> my
>
> Please remember that the people creating the software are also doing
> it for "free in their spare time". It is getting better all the time
> too, and you can help with that too, by making it clear specifically
> what sorts of problems you are having, and how you think things could
> be improved. Ranting about things "not being acceptable" isn't going
> to get you far though.
>
> Rob
>
> -- 
> Robert Hart
>
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 03 May 2007 17:05:43 +0100
> From: "Robert (Jamie) Munro" <rjmunro at arjam.net>
> Subject: Re: [OSM-talk] live rendering
> To: Frederik Ramm <frederik at remote.org>
> Cc: Talk Openstreetmap <talk at openstreetmap.org>
> Message-ID: <463A0857.2030909 at arjam.net>
> Content-Type: text/plain; charset=UTF-8
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Frederik Ramm wrote:
>> Hi,
>>
>>> The
>>> problem is that it can't run on the current database, mainly because the
>>> current database isn't spatially indexed. Schuler is working on what I
>>> think will be the answer to all our problems - migrating the current
>>> database structure to PostGIS without changing it, and then adding
>>> spatial columns with triggers.
>>
>> Nothing against all that. (As far as I know, the current database *is*
>> spatially indexed, only not in a way suitable for Mapnik.)
>>
>>> These spatial columns are not part of the logical model of the data,
>>> which remains unchanged, they are just added as a sort of index or
>>> cache. Mapnik should be able to work directly from these columns
>>
>> I remember reading Artem complaining about lots of things that our
>> current model has and which upset rendering, e.g. self-intersecting ways
>> and so on. While I can easily believe that these can be fixed in a
>> conversion step, I am not sure whether they can be fixed by adding
>> indexes and not changing the real data.
>>
>> Furthermore, I believe that spatial indexing isn't all; you will
>> probably also need indexes on various metadata aspects. Rendering a
>> level-7 tile "on the fly" would require transferring more than a
>> gigabyte of data from the database if you only have "bounding box"
>> requests. You will need to have the ability to query the database for
>> exactly what you want to render - and even then, say if you only use
>> motorways and cosatlines, the amount of data retrieved for processing
>> will be very large.
>
> We could use triggers to assign an importance field to features. The
> importance would roughly correspond to the zoom level the feature is
> visible on. We could then use partial indexes to speed pulling data from
> them. Alternatively, we could use triggers to maintain low-zoom summary
> features in their own database tables.
>
>>> you won't have to wait a week for Planet.osm, or even an hour for T at H to
>>> see your changes, they will be rendered instantly once you have uploaded
>>> them.
>>
>> Provided that a proper mechanism for invalidating tiles exists, and
>> provided that both database and rendering engine really deliver the
>> power to re-render a level-7 tile every time something in its area has
>> changed.
>
> It's possible that we may not be able to render level 7 tiles on the fly
> - - but so what. We could render high zoom levels on the fly (say 12 and
> above), and that is what users editing will want to see.
>
>>> As a side effect, the API will run a lot quicker because it will
>>> be able to use the spatial columns to speed queries.
>>
>> Unless, of course, the database is bogged down by delivering data to the
>> rendering viewer.
>
> It's currently bogged down delivering data to a distributed network of
> T at H clients :-)
>
>> I would be extremely happy to see a "live renderer", don't get me wrong
>> - but I am not exactly over-optimistic that we'll have it within the
>> next half year.
>
> It is a solved problem. See http://labs.metacarta.com/osm. We just have
> to get updates into it as they happen, rather than once a week. PostGres
> triggers (or rules) are almost certainly capable of doing this.
>
> I'm sure improvements can be made. I'm mainly posting this stuff to
> avoid duplication of effort. Schuler is working on moving the DB to
> PostGIS and writing the triggers. I'm hoping to pick up his DB and port
> the rails port to it at the Oxford Dev meeting on Saturday (unless he
> does that first). The next stage is to integrate Schuler's DB layout
> with Mapnik.
>
> Robert (Jamie) Munro
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFGOghVz+aYVHdncI0RAr9aAKDlU9/tcfyBy/78iItunv5izUrgGwCg31WC
> 6FOvhLD7S9hj331sw/+cc+M=
> =0Ow7
> -----END PGP SIGNATURE-----
>
>
>
> ------------------------------
>
> Message: 3
> Date: Thu, 3 May 2007 17:11:46 +0100
> From: Nick Whitelegg <Nick.Whitelegg at solent.ac.uk>
> Subject: [OSM-talk] Karlsruhe area (was Black Forest)
> To: talk at openstreetmap.org
> Message-ID:
> <OFEAB0FEAF.B4887634-ON802572D0.00588324-802572D0.0058F7A5 at solent-university.ac.uk>
>
> Content-Type: text/plain; charset="US-ASCII"
>
>>Meanwhile there are at least two people mapping in the Freiburg area.
>>I'm in Karlsruhe, which is at the northmost beginning of the black
>>forest. It takes 30' by train to get to Bad Herrenalb or even Baden
>>Baden.
>
> Maybe the Karlsruhe area would be the best place, as there seem to be
> quite a few OSM people there already and it would be good to meet
> everyone.
>
> What's the north end of the Black Forest like, scenically, by the way? Is
> it similar to the south? My memories of the Freiburg area were of forested
> mountains with steep sided valleys, and peaks at around 1000-1500 metres.
>
> That's something I was trying to find out on the internet by the way. If
> you're into walking, it's hard enough finding a proprietary map of an area
> showing contours and paths, never mind a free map... hence there's an
> obvious opening for OSM there :-) I had the same problem a year or two ago
> when I was trying to find out walking maps of parts of Canada and the USA.
>
> What time of year would be best? I can do
>
> * late July (16th onwards)
> * all of August
> * late September (15th onwards)
>
> Nick
>
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 3 May 2007 11:18:58 -0700
> From: Schuyler Erle <schuyler at nocat.net>
> Subject: Re: [OSM-talk] live rendering
> To: "Robert (Jamie) Munro" <rjmunro at arjam.net>
> Cc: Talk Openstreetmap <talk at openstreetmap.org>
> Message-ID: <20070503181858.GN2612 at vishnu.tridity.org>
> Content-Type: text/plain; charset=us-ascii
>
> * On  3-May-2007 at  9:07AM PDT, Robert (Jamie) Munro said:
>>
>> It is a solved problem. See http://labs.metacarta.com/osm. We just have
>> to get updates into it as they happen, rather than once a week. PostGres
>> triggers (or rules) are almost certainly capable of doing this.
>
> Luka Frelih points out to me via private email that PostgreSQL also
> supports notifications, which offers interesting possibilities. I
> would be very very glad to work on setting up the labs.metacarta
> tiling system to pull live updates from the main server.
>
>> I'm sure improvements can be made. I'm mainly posting this stuff to
>> avoid duplication of effort. Schuler is working on moving the DB to
>> PostGIS and writing the triggers. I'm hoping to pick up his DB and port
>> the rails port to it at the Oxford Dev meeting on Saturday (unless he
>> does that first).
>
> I don't think I'm going to get to it sooner, but my working materials
> are online at http://freemap.in/~sderle/pg_osm/. Don't forget to
> UTF8sanitize your planet.osm before piping it to import_osm.py.
>
>> The next stage is to integrate Schuler's DB layout with Mapnik.
>
> This is going to be a bit tricky, since Mapnik (I think) expects a
> much flatter database structure than the Rails port uses. I have a
> scheme in mind for mimicking this structure on the fly with a mesh of
> views, rules, and indexes built on procedural functions but I haven't
> tested it and can't yet say how it will perform. Certainly we can
> maintain summary tables with triggers if it comes to that; Christopher
> Schmidt and I have tested this on our own databases and it does work.
>
> One thing at a time, though :)
>
> SDE
>
>
>
> ------------------------------
>
> Message: 5
> Date: Thu, 03 May 2007 20:27:07 +0200
> From: Frederik Ramm <frederik at remote.org>
> Subject: Re: [OSM-talk] live rendering
> To: "Robert (Jamie) Munro" <rjmunro at arjam.net>
> Cc: Talk Openstreetmap <talk at openstreetmap.org>
> Message-ID: <463A297B.9010907 at remote.org>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> Hi,
>
>> We could use triggers to assign an importance field to features.
>
> The more you use triggers, the more you slow down a simple update with
> tasks that are in fact the first step of a rendering process. I still
> maintain it would be a much more solid alternative to invest work in a
> way of keeping a live copy of the data, where you can run triggers to
> your heart's content without delaying changes or insertions of new data
> to the central repository.
>
> But this is an opinion built from theory, and from having seen
> considerable harm done by triggers in the wrong hands ;-) - I cannot say
> how complicated the required operations will be, and maybe Postgres even
> has a way to execute triggers asynchronously without delaying the
> operation that fired the trigger.
>
> > Alternatively, we could use triggers to maintain low-zoom summary
>> features in their own database tables.
>
> I was thinking of replication one level above the database - let the API
> write a kind of "spool file" where it stuffs things in addition to
> writing them to the database. Others can then use that spool file to
> feed their own databases. Again, in my eyes triggers are unsuitable for
> handling the intensity of write access we are seeing, but I haven't
> *tried* and I'd be happy for anyone to prove the contrary.
>
>>> I would be extremely happy to see a "live renderer", don't get me wrong
>>> - but I am not exactly over-optimistic that we'll have it within the
>>> next half year.
>>
>> It is a solved problem. See http://labs.metacarta.com/osm. We just have
>> to get updates into it as they happen, rather than once a week. PostGres
>> triggers (or rules) are almost certainly capable of doing this.
>
> When saying "live renderer", I was referring to something that uses live
> data. The solved part is creating PNGs from specially prepared data on
> the fly; the unsolved part is special-preparing our data in real time. I
> see all sorts of problems looming, but maybe I am just overly skeptical.
> If you get this to work, it would surely be a very big step forward, and
> you'd have my respect.
>
> Bye
> Frederik
>
> -- 
> Frederik Ramm  ##  eMail frederik at remote.org  ##  N49?00.09' E008?23.33'
>
>
>
> ------------------------------
>
> Message: 6
> Date: Thu, 3 May 2007 19:59:51 +0100
> From: OJW <streetmap at blibbleblobble.co.uk>
> Subject: Re: [OSM-talk] riverbanks, way, segs, corrruption, user
> experience, traction.
> To: talk at openstreetmap.org
> Message-ID: <200705031959.52448.streetmap at blibbleblobble.co.uk>
> Content-Type: text/plain;  charset="iso-8859-1"
>
> On Thursday 03 May 2007 16:29, Robert Hart wrote:
>> it really can't be all that bad. You delete the ways, but still have
>> all the segments left over. A few hours perhaps for a really long
>> river - but why were you trying to do all that at once without testing
>> things out on a small section? If you try and run before you can walk,
>> then problems are fairly likely.
>
> Someone on IRC pointed out a trick for deleting a way, its segments and 
> its
> nodes in JOSM:
>
> 1) select the way
> 2) drag it out to sea
> 3) use the rectangle-select to delete all the components
>
>
>
>
>
> ------------------------------
>
> Message: 7
> Date: Thu, 3 May 2007 21:45:08 +0100
> From: "David Earl" <david at frankieandshadow.com>
> Subject: [OSM-talk] Tourlaville
> To: "OSM" <talk at openstreetmap.org>
> Message-ID: <NBBBLBMPJOKFJHGBJPFOAENKHEAA.david at frankieandshadow.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Does anyone know anything about Tourlaville (is_in=Europe, France,
> Basse-Normandie, Manche). Is there someone called Alban on this list who 
> may
> know something about this?
>
> There are no less than 107 separate nodes with
> place=town,place_name=Tourlaville, which seems a rather excessive way of
> describing somewhere!
>
> Looking at them in JOSM, it seems that about half the highway-connector
> nodes have hadd these tags (and postcode=50110) applied to them in the 
> area.
>
> Does anyone know if these were there for a reason? If not, I'll go ahead 
> and
> remove these extraneous tags.
>
> David
>
>
>
>
>
> ------------------------------
>
> Message: 8
> Date: Thu, 3 May 2007 23:29:36 +0200
> From: "Martijn van Oosterhout" <kleptog at gmail.com>
> Subject: Re: [OSM-talk] live rendering
> To: "Frederik Ramm" <frederik at remote.org>
> Cc: Talk Openstreetmap <talk at openstreetmap.org>
> Message-ID:
> <2fc2c5f10705031429y68698a0n7e786796304281ad at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 5/3/07, Frederik Ramm <frederik at remote.org> wrote:
>> The more you use triggers, the more you slow down a simple update with
>> tasks that are in fact the first step of a rendering process. I still
>> maintain it would be a much more solid alternative to invest work in a
>> way of keeping a live copy of the data, where you can run triggers to
>> your heart's content without delaying changes or insertions of new data
>> to the central repository.
>
> Umm, on average the data is going to be read far more often than it's
> going to be written so the cost of the trigger is marginal. Certainly
> cheaper than doing it each time you render. Also postgres is lock free
> (readers don't block writers, writers don't block readers) so there's
> no real downside to having the update take a little longer, it doesn't
> affect the number of concurrent queries.
>
>> But this is an opinion built from theory, and from having seen
>> considerable harm done by triggers in the wrong hands ;-) - I cannot say
>> how complicated the required operations will be, and maybe Postgres even
>> has a way to execute triggers asynchronously without delaying the
>> operation that fired the trigger.
>
> Technically, yes it can. In practice it's usually better to have the
> trigger update a table immediatly and have a seperate asyncronous
> process handle the hard work. I havn't measured it, but I beleive
> having a trigger calculate the bounding box straight away on insert is
> cheaper than updating the row again later, not least because you save
> a several disk writes and get less wasted disk space.
>
> Yes, triggers can be evil when used poorly, but on postgres they are
> usually more efficient than the alternatives.
>
> Have a nice day,
> -- 
> Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/
>
>
>
> ------------------------------
>
> Message: 9
> Date: Thu, 3 May 2007 22:39:22 +0100
> From: 80n <80n80n at gmail.com>
> Subject: [OSM-talk] Richmond on Thames Mapping Party - Accommodation
> To: OSM <talk at openstreetmap.org>
> Message-ID:
> <8fcd02310705031439i4a3ea55bibea96d8b473d50f6 at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> The Fill the Gap Mapping party in Richmond is about 6 weeks away.
>
> Richmond is one of the posher London boroughs but don't think that means
> that it has to be expensive to visit.
>
> I've just added some information about accommodation options for anyone
> thinking about staying over.  There are nearby B&Bs that cost as little as
> ?30 per night.  The mapping party venue is also a hotel, if convenience is
> your thing.
>
> For anyone planning to come by train, Richmond/Kew is very well connected
> with trains from Waterloo and Reading.  Its also on the District Line and
> the North London line so wherever you live in London or South East England
> it should be pretty easy to get to.
>
> If you need to know more or want to come then please sign up here:
> http://wiki.openstreetmap.org/index.php/Richmond-on-Thames
>
> Etienne
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: 
> http://lists.openstreetmap.org/pipermail/talk/attachments/20070503/e92cd2b1/attachment.htm
>
> ------------------------------
>
> _______________________________________________
> talk mailing list
> talk at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
>
>
> End of talk Digest, Vol 33, Issue 11
> ************************************
>