[OSM-talk] Torrents for planet.osm.bz2 are available (again)

Matija Nalis mnalis-osmtorrent-ml at voyager.hr
Sat Dec 25 18:34:50 GMT 2010


On Fri, 17 Dec 2010 00:11:04 +0100 (CET), Stefan de Konink <stefan at konink.de> wrote:
> Thanks for picking this project up :)

I'm glad if people like it and use it !

BTW, there is now RSS feed at:
http://osm-torrent.torres.voyager.hr/files/rss.xml 

so interested parties (and those willing to help to donate some bandwith to
build better torrent swarm) can subscribe to.  Your feedback of how well it
works is also most welcome for this proof-of-conecept project.

As I said before, it would however be very nice if OSMF would pick this up
and setup on their server planet.openstreetmap.org (it is very easy
procedure, and it would spare bandwith and completely remove [up to 48h] 
delay that is currently imposed)

So some lobbying at OSMF to implement it would be most welcome :-)

All scripts are available at http://osm-torrent.torres.voyager.hr/ and I'll
help in any way I can to help implement it if needed (and the scripts
are really very simple and only tiny amount of work is needed to implement
those!)

> Hopefully the PBF's can get seeded as well :)

I'd love to, but:

a) I'm lacking in disk space in machine on which I have enough bandwidth 
   (and I can't add disks in it). This is the biggest showstopper ATM. 
   And on the other machine which has enough disk space I can't spare the
   bandwidth for few dozen GB a day (or more).

b) only PBFs I've been able to find are the ones at:
   http://download.geofabrik.de/osm/europe/ which has few problems:

   b1) torrent is not that big a gain on small files, it really shines on
       bigger ones (and geofabrik mostly provides country extracts which can
       be as small as one megabyte). It might be however possible to create
       multi-file torrent which contains all extract, and people could then
       cherry-pick what they need, which would solve most of this problem.

   b2) webseed feature of torrent files really expects that files don't
       change (the ones at planet.openstreetmap.org don't as they have date
       in filename, however the ones at geofabrik are overwriten daily
       AFAICT).  This could however easily be fixed with a change if file
       naming policy at geofabrik (and/or other PBF provider). This could
       also be worked around if I had solution for (a).

   b3) torrents (especially delayed ones, as are any wh are not created
       directly at the same place of main PBF creation - like mine site)
       prefer files that change at slower rythm (weekly planets are fine,
       daily planets might be too frequent). Although this could also be
       fine if you can host dozen last versions at the same time - which 
       also could be mostly worked around - if I had solution for (a)

Most if not all of those issues would be alleviated or completely removed if
geofabrik (and/or OSMF) would decide to implement those torrents directly at
their servers.

-- 
Opinions above are GNU-copylefted.




More information about the talk mailing list