[OSM-dev] planet.osm.org is verry slow

Simon Poole simon at poole.ch
Sat Jan 25 20:43:54 UTC 2020


Am 25.01.2020 um 21:12 schrieb marc marc:
> is there a convenient way to update a pbf to exactly the same diff as
> osm.org? and/or add the state.txt of the pbf produced on osm.org?
> this would allow to have the same pbf mirrored on other sites without
> having to download and/or encode the same thing several times

The answer is likely no, because I don't believe there is a guarantee
that the planet dump contains exactly the commits of a certain set of
diffs, so I suspect you are asking for something that the planet dump
itself doesn't actually provide  (Matt Amos would be able to give more
information on that, see https://github.com/zerebubuth/planet-dump-ng).
But that doesn't matter in a practical sense, as you will always roll
back a bit the 1st time you start applying diffs and from then on your
planet will exactly contain a diff or not.

There is perhaps an argument to be made that actual mirrors should make
a binary copy. In any case there is no argument to be made for the
~1'000 downloads that start shortly every week after the plant dump has
been produced, and of which for weird reasons 25% are actually
downloading the compressed XML file (which is 30% larger).

Simon

>
> Le 25.01.20 à 18:34, marc tobias a écrit :
>> Wolfram, who asked about the bandwidth, is already running a mirror
>> https://download.bbbike.org/osm/planet/
>> (listed on https://wiki.openstreetmap.org/wiki/Planet.osm#Downloading)
>> so downloading it every week is likely trying to update his mirror.
>>
>> marc tobias
>>
>>> Message: 2
>>> Date: Sat, 25 Jan 2020 11:46:20 +0100
>>> From: Simon Poole <simon at poole.ch>
>>> To: dev at openstreetmap.org
>>> Subject: Re: [OSM-dev] planet.osm.org is verry slow
>>> Message-ID: <4f5c3f2c-f3c9-0545-0dd2-98df390e4948 at poole.ch>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Yes, there are now bandwidth restrictions on connections to
>>> planet.openstreetmap.org because the usage had skyrocketed and outgoing
>>> bandwidth was completely saturated.
>>>
>>> But the obvious question is: why are you even trying to download the
>>> full planet twice within a fortnight? Particularly given the file can
>>> easily be updated in situ.
>>>
>>> Simon
>> _______________________________________________
>> dev mailing list
>> dev at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/dev
>>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20200125/b8f9e8fe/attachment-0001.sig>


More information about the dev mailing list