[OSM-dev] Extracting Great Britain and Ireland from planet.osm

[Ludwig M Brinckmann]

I think you misunderstood me. I perfectly understand the reasons why the API
gives the 400 error.

What I thought was 'insane' however was the attitude that my desire to
'download the whole of London' was somehow not justified rather than just
pointing out a temporary shortcoming of the software. Just because it did
not work and there was no trivial workaround (apart from download the whole
planet, install ruby, install another DB, configure a few scripts...) did
not mean that my request was somehow outlandish. Remember, 'the customer is
always right' ;-)

But with these scripts I think we will get a very good workaround that will
reduce workload on the server and give people what they want: easy access to
open data. Best of both worlds really. Over and out.

Ludwig

On 3/13/07, Nick Black <nickblack1 at gmail.com> wrote:
>
> On 3/13/07, Ludwig M Brinckmann <ludwigbrinckmann at gmail.com> wrote:
> > Thank you, thank you, thank you.
> >
> > While I have not tried it yet, scripts like this will make OSM usable
> again
> > for those who actually want to use the data (wasn't that one of the main
> > goals?) When - at a recent mapping event - I was told by one of the key
> OSM
> > people 'Who wants to download the whole of London?' when I complained
> that
> > the API does not work for this anymore I seriously began to doubt the
> > project's sanity. And stopped using it -- life is a bit too short.
>
> The "sanity" of the decision to cap downloads based on the number of
> features was  that the API was being hammered by large request and
> consequently was not functioning.  The "insanity" would have been to
> ignore the situation to the point where only those people who happen
> to have a script on cron at 4am can download anything, whilst someone
> trying to do a bit of mapping at lunchtime, can't get the applet
> download any data because the API is being nailed.
>
>
>
> >
> > Two suggestions:
> > 1) have this script generate the gb/irl data on the server and store it
> next
> > to the planet.osm. This will seriously cut down the amount of data have
> to
> > download and make it easier to work with up to date data. I assume there
> > will be a lot of interest in the UK/IRL/Germany only data.
> > 2) start a collection of interesting polygons (cities, counties, regions
> > etc). While it might not make sense producing the cut down data files
> for
> > all these polygons, it will help people get started.
> >
> > Ludwig
> >
> >
> > On 3/13/07, Keith Sharp <kms at passback.co.uk> wrote:
> > > Hello,
> > >
> > > Following on from last weeks discussion about reducing the size of the
> > > planet.osm data set for particular use cases I have modified a script
> > > written by Frederik Ramm that extracted data for Germany so that it
> > > extracts data for Great Britain, Ireland, the Channel Islands, the
> Isles
> > > of Scilly, St Kilda, Orkney, and Shetland.  The polygon I used can be
> > > seen:
> > >
> > >          http://www.passback.co.uk/maps/gbirl.html
> > >
> > > You can download the script from:
> > >
> > >         http://www.passback.co.uk/maps/extract-gb-irl.pl
> > >
> > > You run the script as follows:
> > >
> > >         ./extract-gb-irl.pl < planet-070307.osm > gb-irl.osm
> > >
> > > On my system a run took just over 5 minutes wall clock time, I have
> not
> > > looked at memory usage in any detail, but it looked about 50MB max
> > > resident size in top (not the best measure, I know).  The uncompressed
> > > file is reduced from 3.40Gb to 0.22Gb, bzip2 compressed size is 16Mb.
> > >
> > > I have done a little checking of the data and it looks like it is
> > > working as expected, but I would appreciate if others could check as
> > > well.  Would it be possible to get the script added to the OSM SVN,
> > > preferably next to Frederiks script.
> > >
> > > A final question, I created the visualisation of the polygon in Google
> > > Maps because I could not see how to do the same thing using the OSM
> > > infrastructure, am I correct in thinking that there is no public API
> for
> > > OSM at this stage?
> > >
> > > Thanks,
> > >
> > > Keith.
> > >
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev at openstreetmap.org
> > >
> > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
> > >
> >
> >
> > _______________________________________________
> > dev mailing list
> > dev at openstreetmap.org
> > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
> >
> >
>
>
> --
> Nick Black
> --------------------------------
> http://www.blacksworld.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070313/f58caf3a/attachment.html>