[Openstreetmap] How do we handle large amount of GPS points?

SteveC steve at fractalus.com
Fri Nov 19 20:18:56 GMT 2004


* Petter Reinholdtsen (pere at hungry.com) wrote:
> So, I managed to get in touch with one linux user in Oslo, which has
> been driving around in the area with a GPS and a laptop for quite some
> time now.  He got >7 million GPS coordinates, which he uses to create
> a bitmap map of Oslo.  He was very positive to the idea of sharing
> this data set with us, and I got a copy of his current database.

thats fantastic!

1/

So I've written a skelaton XML-RPC interface to openstreetmap which I'm
documenting here:

http://bat.vr.ucl.ac.uk/steve/wiki/index.php/XmlRpcInterface

so that anyone can start writing tools to do real actual stuff. I've
moved over the applet to use this (mostly, some bugs with
org.apache.xmlrpc.applet break things).

Its all in the subversion tree with a test program to login and add a
point. Hopefully this is good enough for perlites, pythonistas and so on
begin using it with whatever format/language etc you like. I know there
are many functions to add, let me know whats important to you and I'll
add it (see below for a little discussion). If you could add code you
develop to the subversion tree for others that'd be cool.

2/

login accounts. I'll write something to create your own accounts soon,
till then if you want one drop me an email

3/

website. the machine will get php this weekend I'm told, so I'll move
the wiki etc there too. I'll drop MovableType for WordPress.

> I guess my question is really, how should we handle such wast amount
> of points?  Steve, do you want 7 million points from Norway in your
> database?  Ready to handle it?  How do you want it?

Yes, I want it :-)

The machine is RAIDed so data should be safe. The answer on how to get
it in the database is dependant on the next bit:

> CREATE TABLE track (
>   ID bigint(8) NOT NULL auto_increment,
>   time datetime NOT NULL default '0000-00-00 00:00:00',
>   lat decimal(8,6) NOT NULL default '0.000000',
>   lon decimal(8,6) NOT NULL default '0.000000',
>   alt int(4) NOT NULL default '0',
>   fix smallint(1) NOT NULL default '0',
>   speed decimal(5,2) NOT NULL default '0.00',
>   trackfile varchar(150) NOT NULL default '',
>   PRIMARY KEY  (ID),
>   UNIQUE KEY trackfile_2 (trackfile,lat,lon)
> ) TYPE=MyISAM;
> 
> fix is 2 for 2D (3 satellites) and 3 for 3D (>=4 satellites).  I'll
> import it into PostgreSQL and do some processing there.

Now, my table looks like lat,lon,alt,time,userid

The question is, what other fields are important to people? FWIW my
views are:

speed -  not needed
track keys - not needed
fix - you guys might want that?

let me know and I'll update the SQL, the XMLRPC and the wiki

That being decided you could write a quick perl thing to call the RPC
interface with your points, or I could do it locally...

have fun,

SteveC steve at fractalus.com http://www.fractalus.com/steve/




More information about the talk mailing list