[OSM-dev] Binary OSM; the first pass encoder

[Stefan de Konink]

On Wed, 12 Nov 2008, Tomas Kolda wrote:

> Integer has better precision if you are far from zero, because floats
> use only 24 bits for significant, other bits is for sign and exponent.
> So 32 bit integer is better if you know extends (-180, .... 180). Min
> and max of int is -2.1e9 to 2.1e9. So if you multiply longitude and
> latitude by 1e7 you get almost maximum from 32bit int in all range (from
> -1.8e9 to 1.8e9).
>
> It is also faster to process. And if you are doing some difference
> compression, it is easier to do with ints.

Integers are only a better precision if all the bits are used. Thus only
improve the situation if a multiplication with 2^32 / 360 is used.

Now the minor detail is that this operation has an error of 10^-7, but the
more preciese the input the better the storage resolution.

89 * (2^32 / 360)
1061811359.28889 ---> 1061811359

1061811359 / (2^32 / 360) ---> 1061811359 * 360  /  2^32 (bit shift)
88.9999999757856


So what you suggest is also a naive method; but that method just killed
the most significant 10^-8 digit.

Basically it says;

360.0000000 =/ 3600000000


Now what was mentioned before; is the hard cut of a problem? Not if the
input is typically only significant till 10^-6. Should we overmodel it
now? Probably not until we have data that is that significant.


Stefan